Re: X32 psABI status update

[H.J. Lu]

On Mon, Mar 21, 2011 at 1:20 AM, Michael Matz  wrote:
> Hi,
>
> On Sun, 20 Mar 2011, H.J. Lu wrote:
>
>> I don't think it will help x32 and I think it will make it harder to add
>> x32 support. I still want to see a real usage before I add it.
>
> % cat real-world.c
> /* intptr_t; what's that? */
> union space_saving_htab_element {
>  void *generic_pointer;
>  /* Usually we need a long for a pointer, but I just figured out
>     that on x32 an int is enough and smaller.  My program
>     now needs half as much memory, supi!  */
> #ifdef __x32__
>  unsigned int as_number;
> #else
>  unsigned long as_number;
> #endif
> };

That is the wrong way to support x32.  You should remove "#ifdef __x32__",
which only shows __x32__ shouldn't be used/needed.

>
> Ciao,
> Michael.
> PS: Of course you and I wouldn't write such code, but Mikes point was that
> there might be some that do.  I could probably construct an example where
> it would matter for real involving inline asm that for some reason has to
> slightly differ depending on x32-ness.
>

I  am still waiting for a real example.

-- 
H.J.

Re: X32 psABI status update

[Michael Matz]

Hi,

On Sun, 20 Mar 2011, H.J. Lu wrote:

> I don't think it will help x32 and I think it will make it harder to add 
> x32 support. I still want to see a real usage before I add it.

% cat real-world.c
/* intptr_t; what's that? */
union space_saving_htab_element {
  void *generic_pointer;
  /* Usually we need a long for a pointer, but I just figured out
 that on x32 an int is enough and smaller.  My program
 now needs half as much memory, supi!  */
#ifdef __x32__
  unsigned int as_number;
#else
  unsigned long as_number;
#endif
};


Ciao,
Michael.
PS: Of course you and I wouldn't write such code, but Mikes point was that
there might be some that do.  I could probably construct an example where 
it would matter for real involving inline asm that for some reason has to 
slightly differ depending on x32-ness.

Re: X32 psABI status update

[H.J. Lu]

On Sun, Mar 20, 2011 at 10:53 PM, Mike Frysinger  wrote:
> On Monday, March 21, 2011 01:35:35 H.J. Lu wrote:
>> On Sun, Mar 20, 2011 at 10:08 PM, Mike Frysinger wrote:
>> > On Thursday, March 17, 2011 01:21:16 H.J. Lu wrote:
>> >> On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger wrote:
>> >> > in looking at the gcc files, it doesnt seem like there's any defines
>> >> > setup to declare x32 directly.  instead, you'd have to do something
>> >> > like: #ifdef __x86_64__
>> >> > # if __SIZEOF_LONG__ == 8
>> >> > /* x86_64 */
>> >> > # else
>> >> > /* x32 */
>> >> > # endif
>> >> > #endif
>> >> >
>> >> > any plans on adding an __x32__ (or whatever) cpp symbol to keep people
>> >> > from coming up with their own special/broken crap ?  or are there some
>> >> > already that i'm not seeing ?
>> >>
>> >> The idea is in most cases, you only need to check __x86_64__ since x32
>> >> and x86-64 are very close.  In some cases, x32 is very different from
>> >> x86_64, like assembly codes on long and pointer, you can check
>> >> __x86_64__ and __LP64__. In glibc, I used a different approach by using
>> >> macros REG_RAX, .., MOV_LP, ADD_LP, SUB_LP and CMP_LP in assembly
>> >> codes.
>> >
>> > while i agree with you in general that this is how people should be doing
>> > things, in practice i often see people fishing around.  education only
>> > goes so far, so if there was an __x32__ define, i feel like people are
>> > more likely to get it right than wrong.
>> >
>> > i dont have any use cases off the top of my head, but i wouldnt be
>> > surprised if the heavy inline assembly people (like the multimedia peeps
>> > e.g. libav) approached it this way.  rather than google for
>> > documentation, look at the cpp output between -m64 and -mx32 and see
>> > what sticks out.  "__x32__" would certainly do that.
>>
>> My point is __x86_64__ version should work for both 64bit and x32. There
>> should no need for a separate x32 version.
>
> yes, and my point is that people often reach for cpp defines rather than
> figure out how to do it right.
>
> i'm not saying that __x32__ should be defined in place of __x86_64__, just
> that when -mx32 is used, __x32__ is additionally defined.  simply as an
> example, what i'm referring to could be accomplished by adding this to the
> *cpp specs:
>        %{mx32:-D__x32__=1}

I don't think it will help x32 and I think it will make it harder to
add x32 support.
I still want to see a real usage before I add it.


-- 
H.J.

Re: X32 psABI status update

[Mike Frysinger]

On Monday, March 21, 2011 01:35:35 H.J. Lu wrote:
> On Sun, Mar 20, 2011 at 10:08 PM, Mike Frysinger wrote:
> > On Thursday, March 17, 2011 01:21:16 H.J. Lu wrote:
> >> On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger wrote:
> >> > in looking at the gcc files, it doesnt seem like there's any defines
> >> > setup to declare x32 directly.  instead, you'd have to do something
> >> > like: #ifdef __x86_64__
> >> > # if __SIZEOF_LONG__ == 8
> >> > /* x86_64 */
> >> > # else
> >> > /* x32 */
> >> > # endif
> >> > #endif
> >> > 
> >> > any plans on adding an __x32__ (or whatever) cpp symbol to keep people
> >> > from coming up with their own special/broken crap ?  or are there some
> >> > already that i'm not seeing ?
> >> 
> >> The idea is in most cases, you only need to check __x86_64__ since x32
> >> and x86-64 are very close.  In some cases, x32 is very different from
> >> x86_64, like assembly codes on long and pointer, you can check
> >> __x86_64__ and __LP64__. In glibc, I used a different approach by using
> >> macros REG_RAX, .., MOV_LP, ADD_LP, SUB_LP and CMP_LP in assembly
> >> codes.
> > 
> > while i agree with you in general that this is how people should be doing
> > things, in practice i often see people fishing around.  education only
> > goes so far, so if there was an __x32__ define, i feel like people are
> > more likely to get it right than wrong.
> > 
> > i dont have any use cases off the top of my head, but i wouldnt be
> > surprised if the heavy inline assembly people (like the multimedia peeps
> > e.g. libav) approached it this way.  rather than google for
> > documentation, look at the cpp output between -m64 and -mx32 and see
> > what sticks out.  "__x32__" would certainly do that.
> 
> My point is __x86_64__ version should work for both 64bit and x32. There
> should no need for a separate x32 version.

yes, and my point is that people often reach for cpp defines rather than 
figure out how to do it right.

i'm not saying that __x32__ should be defined in place of __x86_64__, just 
that when -mx32 is used, __x32__ is additionally defined.  simply as an 
example, what i'm referring to could be accomplished by adding this to the 
*cpp specs:
%{mx32:-D__x32__=1}
-mike


signature.asc
Description: This is a digitally signed message part.

Re: X32 psABI status update

[H.J. Lu]

On Sun, Mar 20, 2011 at 10:08 PM, Mike Frysinger  wrote:
> On Thursday, March 17, 2011 01:21:16 H.J. Lu wrote:
>> On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger  wrote:
>> > in looking at the gcc files, it doesnt seem like there's any defines
>> > setup to declare x32 directly.  instead, you'd have to do something
>> > like: #ifdef __x86_64__
>> > # if __SIZEOF_LONG__ == 8
>> > /* x86_64 */
>> > # else
>> > /* x32 */
>> > # endif
>> > #endif
>> >
>> > any plans on adding an __x32__ (or whatever) cpp symbol to keep people
>> > from coming up with their own special/broken crap ?  or are there some
>> > already that i'm not seeing ?
>>
>> The idea is in most cases, you only need to check __x86_64__ since x32 and
>> x86-64 are very close.  In some cases, x32 is very different from x86_64,
>> like assembly codes on long and pointer, you can check __x86_64__ and
>> __LP64__. In glibc, I used a different approach by using macros REG_RAX,
>> .., MOV_LP, ADD_LP, SUB_LP and CMP_LP in assembly codes.
>
> while i agree with you in general that this is how people should be doing
> things, in practice i often see people fishing around.  education only goes so
> far, so if there was an __x32__ define, i feel like people are more likely to
> get it right than wrong.
>
> i dont have any use cases off the top of my head, but i wouldnt be surprised
> if the heavy inline assembly people (like the multimedia peeps e.g. libav)
> approached it this way.  rather than google for documentation, look at the cpp
> output between -m64 and -mx32 and see what sticks out.  "__x32__" would
> certainly do that.

My point is __x86_64__ version should work for both 64bit and x32. There should
no need for a separate x32 version.


-- 
H.J.

Re: X32 psABI status update

[Mike Frysinger]

On Thursday, March 17, 2011 01:21:16 H.J. Lu wrote:
> On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger  wrote:
> > in looking at the gcc files, it doesnt seem like there's any defines
> > setup to declare x32 directly.  instead, you'd have to do something
> > like: #ifdef __x86_64__
> > # if __SIZEOF_LONG__ == 8
> > /* x86_64 */
> > # else
> > /* x32 */
> > # endif
> > #endif
> > 
> > any plans on adding an __x32__ (or whatever) cpp symbol to keep people
> > from coming up with their own special/broken crap ?  or are there some
> > already that i'm not seeing ?
> 
> The idea is in most cases, you only need to check __x86_64__ since x32 and
> x86-64 are very close.  In some cases, x32 is very different from x86_64,
> like assembly codes on long and pointer, you can check __x86_64__ and
> __LP64__. In glibc, I used a different approach by using macros REG_RAX,
> .., MOV_LP, ADD_LP, SUB_LP and CMP_LP in assembly codes.

while i agree with you in general that this is how people should be doing 
things, in practice i often see people fishing around.  education only goes so 
far, so if there was an __x32__ define, i feel like people are more likely to 
get it right than wrong.

i dont have any use cases off the top of my head, but i wouldnt be surprised 
if the heavy inline assembly people (like the multimedia peeps e.g. libav) 
approached it this way.  rather than google for documentation, look at the cpp 
output between -m64 and -mx32 and see what sticks out.  "__x32__" would 
certainly do that.
-mike


signature.asc
Description: This is a digitally signed message part.

Re: X32 psABI status update

[Richard Henderson]

On 03/16/2011 10:21 PM, H.J. Lu wrote:
> On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger  wrote:
>> On Wednesday, March 16, 2011 08:39:57 H.J. Lu wrote:
>>> On Tue, Mar 15, 2011 at 10:24 PM, Mike Frysinger wrote:
 so we get back to my original e-mail:
are you getting a unique host tuple for this ?  or are you
 extending x86_64-linux-gnu ?  so the only way of knowing which ABI is to
 check for the output of the compiler+compiler flags ?
>>>
>>> As I said, the target is x86_64- linux-gnu and you just add -mx32 to
>>> CFLAGS. The x86_64- linux-gnu binutils and GCC support  x32.
>>
>> ok, took long enough, but that answers most things.  your usage of "x32-"
>> prefixed binaries in the documentation seems to imply a lot more than the 
>> fact
>> you just picked those locally to avoid system collisions.  this isnt a wiki
>> page, otherwise i'd clean things up for you.
> 
> Any suggestion how to create a wiki page for x32?

Hanging it off of gcc.gnu.org/wiki wouldn't be a bad idea, imo.


r~

Re: X32 psABI status update

[H.J. Lu]

On Wed, Mar 16, 2011 at 10:45 PM, Mike Frysinger  wrote:
> On Thursday, March 17, 2011 01:21:16 H.J. Lu wrote:
>> On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger  wrote:
>> > ok, took long enough, but that answers most things.  your usage of "x32-"
>> > prefixed binaries in the documentation seems to imply a lot more than the
>> > fact you just picked those locally to avoid system collisions.  this
>> > isnt a wiki page, otherwise i'd clean things up for you.
>>
>> Any suggestion how to create a wiki page for x32?
>
> seems like the sites.google.com documentation says it includes wiki support.
>        http://sites.google.com/site/projectwikitemplate_en/
>
> ive never used google sites before, so i dont know how to actually enable it
> on the admin side of things.

I will take a look.

>> > in looking at the gcc files, it doesnt seem like there's any defines
>> > setup to declare x32 directly.  instead, you'd have to do something
>> > like: #ifdef __x86_64__
>> > # if __SIZEOF_LONG__ == 8
>> > /* x86_64 */
>> > # else
>> > /* x32 */
>> > # endif
>> > #endif
>> >
>> > any plans on adding an __x32__ (or whatever) cpp symbol to keep people
>> > from coming up with their own special/broken crap ?  or are there some
>> > already that i'm not seeing ?
>>
>> The idea is in most cases, you only need to check __x86_64__ since x32 and
>> x86-64 are very close.  In some cases, x32 is very different from x86_64,
>> like assembly codes on long and pointer, you can check __x86_64__ and
>> __LP64__. In glibc, I used a different approach by using macros REG_RAX,
>> .., MOV_LP, ADD_LP, SUB_LP and CMP_LP in assembly codes.
>
> arm/mips/ppc sets up explicit ABI defines to clearly differentiate between
> things.  while __LP64__ should work here, it seems like a poor substitute.
> how about builtin_define("__X32__") ?  or __ABI_X32__ ?  doesnt seem like i386
> has a standard in this regard to piggy off of.

If you can show me some real examples how __x32__ will be used, I will
take a look.

Thanks.

-- 
H.J.

Re: X32 psABI status update

[Mike Frysinger]

On Thursday, March 17, 2011 01:21:16 H.J. Lu wrote:
> On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger  wrote:
> > ok, took long enough, but that answers most things.  your usage of "x32-"
> > prefixed binaries in the documentation seems to imply a lot more than the
> > fact you just picked those locally to avoid system collisions.  this
> > isnt a wiki page, otherwise i'd clean things up for you.
> 
> Any suggestion how to create a wiki page for x32?

seems like the sites.google.com documentation says it includes wiki support.
http://sites.google.com/site/projectwikitemplate_en/

ive never used google sites before, so i dont know how to actually enable it 
on the admin side of things.

> > in looking at the gcc files, it doesnt seem like there's any defines
> > setup to declare x32 directly.  instead, you'd have to do something
> > like: #ifdef __x86_64__
> > # if __SIZEOF_LONG__ == 8
> > /* x86_64 */
> > # else
> > /* x32 */
> > # endif
> > #endif
> > 
> > any plans on adding an __x32__ (or whatever) cpp symbol to keep people
> > from coming up with their own special/broken crap ?  or are there some
> > already that i'm not seeing ?
> 
> The idea is in most cases, you only need to check __x86_64__ since x32 and
> x86-64 are very close.  In some cases, x32 is very different from x86_64,
> like assembly codes on long and pointer, you can check __x86_64__ and
> __LP64__. In glibc, I used a different approach by using macros REG_RAX,
> .., MOV_LP, ADD_LP, SUB_LP and CMP_LP in assembly codes.

arm/mips/ppc sets up explicit ABI defines to clearly differentiate between 
things.  while __LP64__ should work here, it seems like a poor substitute.  
how about builtin_define("__X32__") ?  or __ABI_X32__ ?  doesnt seem like i386 
has a standard in this regard to piggy off of.
-mike


signature.asc
Description: This is a digitally signed message part.

Re: X32 psABI status update

[H.J. Lu]

On Wed, Mar 16, 2011 at 7:57 PM, Mike Frysinger  wrote:
> On Wednesday, March 16, 2011 08:39:57 H.J. Lu wrote:
>> On Tue, Mar 15, 2011 at 10:24 PM, Mike Frysinger wrote:
>> > so we get back to my original e-mail:
>> >        are you getting a unique host tuple for this ?  or are you
>> > extending x86_64-linux-gnu ?  so the only way of knowing which ABI is to
>> > check for the output of the compiler+compiler flags ?
>>
>> As I said, the target is x86_64- linux-gnu and you just add -mx32 to
>> CFLAGS. The x86_64- linux-gnu binutils and GCC support  x32.
>
> ok, took long enough, but that answers most things.  your usage of "x32-"
> prefixed binaries in the documentation seems to imply a lot more than the fact
> you just picked those locally to avoid system collisions.  this isnt a wiki
> page, otherwise i'd clean things up for you.

Any suggestion how to create a wiki page for x32?

> in looking at the gcc files, it doesnt seem like there's any defines setup to
> declare x32 directly.  instead, you'd have to do something like:
> #ifdef __x86_64__
> # if __SIZEOF_LONG__ == 8
> /* x86_64 */
> # else
> /* x32 */
> # endif
> #endif
>
> any plans on adding an __x32__ (or whatever) cpp symbol to keep people from
> coming up with their own special/broken crap ?  or are there some already that
> i'm not seeing ?

The idea is in most cases, you only need to check __x86_64__ since x32 and
x86-64 are very close.  In some cases, x32 is very different from x86_64, like
assembly codes on long and pointer, you can check __x86_64__ and __LP64__.
In glibc, I used a different approach by using macros REG_RAX, .., MOV_LP,
ADD_LP, SUB_LP and CMP_LP in assembly codes.

I added a simple howto for x32 compiling to

https://sites.google.com/site/x32abi/

Thanks.

-- 
H.J.

Re: X32 psABI status update

[Mike Frysinger]

On Wednesday, March 16, 2011 08:39:57 H.J. Lu wrote:
> On Tue, Mar 15, 2011 at 10:24 PM, Mike Frysinger wrote:
> > so we get back to my original e-mail:
> >are you getting a unique host tuple for this ?  or are you
> > extending x86_64-linux-gnu ?  so the only way of knowing which ABI is to
> > check for the output of the compiler+compiler flags ?
> 
> As I said, the target is x86_64- linux-gnu and you just add -mx32 to
> CFLAGS. The x86_64- linux-gnu binutils and GCC support  x32.

ok, took long enough, but that answers most things.  your usage of "x32-" 
prefixed binaries in the documentation seems to imply a lot more than the fact 
you just picked those locally to avoid system collisions.  this isnt a wiki 
page, otherwise i'd clean things up for you.

in looking at the gcc files, it doesnt seem like there's any defines setup to 
declare x32 directly.  instead, you'd have to do something like:
#ifdef __x86_64__
# if __SIZEOF_LONG__ == 8
/* x86_64 */
# else
/* x32 */
# endif
#endif

any plans on adding an __x32__ (or whatever) cpp symbol to keep people from 
coming up with their own special/broken crap ?  or are there some already that 
i'm not seeing ?
-mike


signature.asc
Description: This is a digitally signed message part.

Re: X32 psABI status update

[Andreas Schwab]

Mike Frysinger  writes:

> but it doesnt say what to do for binutils or gcc itself.  obviously you cant 
> use the -mx32 flag when building the cross gcc for the first time since the 
> current native gcc wont support it.

Of course, for a compiler tool the only thing that matters is the
target.

Andreas.

-- 
Andreas Schwab, sch...@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84  5EC7 45C6 250E 6F00 984E
"And now for something completely different."

Re: X32 psABI status update

[H.J. Lu]

On Tue, Mar 15, 2011 at 10:24 PM, Mike Frysinger  wrote:
> On Wednesday, March 16, 2011 00:51:37 H.J. Lu wrote:
>> On Tue, Mar 15, 2011 at 9:30 PM, Mike Frysinger  wrote:
>> > On Wednesday, March 16, 2011 00:17:04 H.J. Lu wrote:
>> >> On Mon, Mar 7, 2011 at 1:33 PM, Mike Frysinger wrote:
>> >> > On Saturday, March 05, 2011 14:08:04 H.J. Lu wrote:
>> >> >> Many x32 bugs are fixed in kernel, glibc, binutils and GCC:
>> >> >>
>> >> >> https://sites.google.com/site/x32abi/
>> >> >>
>> >> >> The major remaining issues are glibc/gcc testsuite failures,
>> >> >> kernel core dump and signal handler unwind.
>> >> >
>> >> > are you getting a unique host tuple for this ?  or are you extending
>> >> > x86_64- linux-gnu ?  so the only way of knowing which ABI is to check
>> >> > for the output of the compiler+compiler flags ?
>> >>
>> >> x32 requires x32 enabled x86-64 kernel.
>> >
>> > ok, but that doesnt answer my question.  what are you passing to --target
>> > ?
>>
>> See:
>>
>> https://sites.google.com/site/x32abi/
>>
>> You configure it as
>>
>> CC="gcc -mx32" ./configure x86_64- linux-gnu
>
> that isnt what the page says.  it does say for glibc to do:
>        CC="x32-gcc -mx32 " CXX="x32-g++ -mx32 " CFLAGS="-O2 -g"  ../configure
>
> but it doesnt say what to do for binutils or gcc itself.  obviously you cant
> use the -mx32 flag when building the cross gcc for the first time since the
> current native gcc wont support it.
>
> the invocation given for glibc implies that the proposed tuple is x32-linux-
> gnu, but the GNU config project doesnt support that tuple and i didnt see
> mention of it in the binutils or gcc sources.
>
> so we get back to my original e-mail:
>        are you getting a unique host tuple for this ?  or are you extending
>        x86_64-linux-gnu ?  so the only way of knowing which ABI is to check 
> for
>        the output of the compiler+compiler flags ?

As I said, the target is x86_64- linux-gnu and you just add -mx32 to CFLAGS.
The x86_64- linux-gnu binutils and GCC support  x32.


-- 
H.J.

Re: X32 psABI status update

[Mike Frysinger]

On Wednesday, March 16, 2011 00:51:37 H.J. Lu wrote:
> On Tue, Mar 15, 2011 at 9:30 PM, Mike Frysinger  wrote:
> > On Wednesday, March 16, 2011 00:17:04 H.J. Lu wrote:
> >> On Mon, Mar 7, 2011 at 1:33 PM, Mike Frysinger wrote:
> >> > On Saturday, March 05, 2011 14:08:04 H.J. Lu wrote:
> >> >> Many x32 bugs are fixed in kernel, glibc, binutils and GCC:
> >> >> 
> >> >> https://sites.google.com/site/x32abi/
> >> >> 
> >> >> The major remaining issues are glibc/gcc testsuite failures,
> >> >> kernel core dump and signal handler unwind.
> >> > 
> >> > are you getting a unique host tuple for this ?  or are you extending
> >> > x86_64- linux-gnu ?  so the only way of knowing which ABI is to check
> >> > for the output of the compiler+compiler flags ?
> >> 
> >> x32 requires x32 enabled x86-64 kernel.
> > 
> > ok, but that doesnt answer my question.  what are you passing to --target
> > ?
> 
> See:
> 
> https://sites.google.com/site/x32abi/
> 
> You configure it as
> 
> CC="gcc -mx32" ./configure x86_64- linux-gnu

that isnt what the page says.  it does say for glibc to do:
CC="x32-gcc -mx32 " CXX="x32-g++ -mx32 " CFLAGS="-O2 -g"  ../configure

but it doesnt say what to do for binutils or gcc itself.  obviously you cant 
use the -mx32 flag when building the cross gcc for the first time since the 
current native gcc wont support it.

the invocation given for glibc implies that the proposed tuple is x32-linux-
gnu, but the GNU config project doesnt support that tuple and i didnt see 
mention of it in the binutils or gcc sources.

so we get back to my original e-mail:
are you getting a unique host tuple for this ?  or are you extending
x86_64-linux-gnu ?  so the only way of knowing which ABI is to check for
the output of the compiler+compiler flags ?
-mike


signature.asc
Description: This is a digitally signed message part.

Re: X32 psABI status update

[H.J. Lu]

On Tue, Mar 15, 2011 at 9:30 PM, Mike Frysinger  wrote:
> On Wednesday, March 16, 2011 00:17:04 H.J. Lu wrote:
>> On Mon, Mar 7, 2011 at 1:33 PM, Mike Frysinger wrote:
>> > On Saturday, March 05, 2011 14:08:04 H.J. Lu wrote:
>> >> Many x32 bugs are fixed in kernel, glibc, binutils and GCC:
>> >>
>> >> https://sites.google.com/site/x32abi/
>> >>
>> >> The major remaining issues are glibc/gcc testsuite failures,
>> >> kernel core dump and signal handler unwind.
>> >
>> > are you getting a unique host tuple for this ?  or are you extending
>> > x86_64- linux-gnu ?  so the only way of knowing which ABI is to check
>> > for the output of the compiler+compiler flags ?
>>
>> x32 requires x32 enabled x86-64 kernel.
>
> ok, but that doesnt answer my question.  what are you passing to --target ?

See:

https://sites.google.com/site/x32abi/

You configure it as

CC="gcc -mx32" ./configure x86_64- linux-gnu


-- 
H.J.

Re: X32 psABI status update

[Mike Frysinger]

On Wednesday, March 16, 2011 00:17:04 H.J. Lu wrote:
> On Mon, Mar 7, 2011 at 1:33 PM, Mike Frysinger wrote:
> > On Saturday, March 05, 2011 14:08:04 H.J. Lu wrote:
> >> Many x32 bugs are fixed in kernel, glibc, binutils and GCC:
> >> 
> >> https://sites.google.com/site/x32abi/
> >> 
> >> The major remaining issues are glibc/gcc testsuite failures,
> >> kernel core dump and signal handler unwind.
> > 
> > are you getting a unique host tuple for this ?  or are you extending
> > x86_64- linux-gnu ?  so the only way of knowing which ABI is to check
> > for the output of the compiler+compiler flags ?
> 
> x32 requires x32 enabled x86-64 kernel.

ok, but that doesnt answer my question.  what are you passing to --target ?
-mike


signature.asc
Description: This is a digitally signed message part.

X32 psABI status update

[H.J. Lu]

Hi,

Almost all x32 bugs in kernel, glibc and binutils are fixed:

https://sites.google.com/site/x32abi/

There are no unexpected failures in glibc testsuite. I am
working on remaining GCC bugs.

-- 
H.J.

Re: X32 psABI status update

[H.J. Lu]

On Mon, Mar 7, 2011 at 1:33 PM, Mike Frysinger  wrote:
> On Saturday, March 05, 2011 14:08:04 H.J. Lu wrote:
>> Many x32 bugs are fixed in kernel, glibc, binutils and GCC:
>>
>> https://sites.google.com/site/x32abi/
>>
>> The major remaining issues are glibc/gcc testsuite failures,
>> kernel core dump and signal handler unwind.
>
> are you getting a unique host tuple for this ?  or are you extending x86_64-
> linux-gnu ?  so the only way of knowing which ABI is to check for the output
> of the compiler+compiler flags ?

x32 requires x32 enabled x86-64 kernel.


-- 
H.J.

Re: X32 psABI status update

[Mike Frysinger]

On Saturday, March 05, 2011 14:08:04 H.J. Lu wrote:
> Many x32 bugs are fixed in kernel, glibc, binutils and GCC:
> 
> https://sites.google.com/site/x32abi/
> 
> The major remaining issues are glibc/gcc testsuite failures,
> kernel core dump and signal handler unwind.

are you getting a unique host tuple for this ?  or are you extending x86_64-
linux-gnu ?  so the only way of knowing which ABI is to check for the output 
of the compiler+compiler flags ?
-mike


signature.asc
Description: This is a digitally signed message part.

Re: X32 psABI status update

[H.J. Lu]

On Sat, Mar 5, 2011 at 11:08 AM, H.J. Lu  wrote:
> Hi,
>
> Many x32 bugs are fixed in kernel, glibc, binutils and GCC:
>
> https://sites.google.com/site/x32abi/
>
> The major remaining issues are glibc/gcc testsuite failures,
> kernel core dump and signal handler unwind.
>

I checked in a kernel patch:

http://git.kernel.org/?p=linux/kernel/git/hjl/linux-2.6.37.y.git;a=commitdiff;h=7fd04de9dca8871772d97928c338d4a2eb315c2b

and a GCC patch:

http://gcc.gnu.org/ml/gcc-patches/2011-03/msg00267.html

to fix signal handler unwind.


-- 
H.J.

X32 psABI status update

[H.J. Lu]

Hi,

Many x32 bugs are fixed in kernel, glibc, binutils and GCC:

https://sites.google.com/site/x32abi/

The major remaining issues are glibc/gcc testsuite failures,
kernel core dump and signal handler unwind.


-- 
H.J.

Re: x32 psABI status update

[H.J. Lu]

More data

* 186.crafty from SPEC CPU 2000 (64bit integer):
  o Intel Core i7
+ ~5% slower than x86-64.
+ ~29% faster than ia32.
+ Sizes in bytes:

   text   databssdec
 hexfilename
 200519  1638810689161285823
139ebfcrafty.32
 172389  1686810697841259041
133621crafty.64
 174723  1656410689161260203
133aabcrafty.x32

  o Intel Atom
+ ~6% slower than x86-64.
+ ~7% faster than ia32.
+ Sizes in bytes:

   text   databssdec
hexfilename
 199339  1638810689161284643 139a23crafty.32
 171157  1686810697841257809 133151crafty.64
 173535  1656410689161259015 133607
crafty.x32

H.J.
---
On Sat, Feb 19, 2011 at 11:23 AM, Xinliang David Li  wrote:
> This shows how important ilp32 is for some applications. It would be good to
> show the impact of larger register set. Candidates are those spec programs
> which are slower with ia32.
>
> David
>
> On Feb 19, 2011 10:14 AM, "H.J. Lu"  wrote:
>> On Fri, Feb 18, 2011 at 8:28 PM, H.J. Lu  wrote:
>>> On Wed, Feb 16, 2011 at 9:45 PM, H.J. Lu  wrote:
 On Wed, Feb 16, 2011 at 11:22 AM, H.J. Lu  wrote:
> Hi,
>
> I updated  x32 psABI draft to version 0.2 to change x32 library path
> from lib32 to libx32 since lib32 is used for ia32 libraries on Debian,
> Ubuntu and other derivative distributions. The new x32 psABI is
> available from:
>
> https://sites.google.com/site/x32abi/home
>

 Initial kernel, glibc and gcc port for x32 psABI draft version 0.2  is
 done:

 1. Kernel should be very stable.  Fedora 14 kernel patch is also
 available.
 2. Need to fix GCC run-time libraries.
 3. Need to fix glibc tests.

 Please check them out and provide feed backs.
>>>
>>> I updated x32 website:
>>>
>>> https://sites.google.com/site/x32abi/
>>>
>>> with "get started" instructions.
>>>
>>
>> I update the x32 website
>>
>> https://sites.google.com/site/x32abi/
>>
>> with data on mcf in SPEC CPU 2K:
>>
>> * 181.mcf from SPEC CPU 2000:
>> o Intel Core i7
>> + ~32% faster than x86-64.
>> + ~1% slower than ia32.
>> + Sizes in bytes:
>>
>> text data bss dec
>> hex filename
>> 12155 336 7036 19527
>> 4c47 mcf.32
>> 12421 664 13568 26653
>> 681d mcf.64
>> 11583 412 7040 19035
>> 4a5b mcf.x32
>>
>> o Intel Atom
>> + ~28% faster than x86-64.
>> + ~0.5% slower than ia32.
>> + Sizes in bytes:
>>
>> text data bss dec
>> hex filename
>> 12059 336 7036 19431
>> 4be7 mcf.32
>> 12341 664 13568 26573
>> 67cd mcf.64
>> 11411 412 7040 18863
>> 49af mcf.x32
>>
>>
>> --
>> H.J.
>



-- 
H.J.

Re: x32 psABI status update

[H.J. Lu]

On Fri, Feb 18, 2011 at 8:28 PM, H.J. Lu  wrote:
> On Wed, Feb 16, 2011 at 9:45 PM, H.J. Lu  wrote:
>> On Wed, Feb 16, 2011 at 11:22 AM, H.J. Lu  wrote:
>>> Hi,
>>>
>>> I updated  x32 psABI draft to version 0.2 to change x32 library path
>>> from lib32 to libx32 since lib32 is used for ia32 libraries on Debian,
>>> Ubuntu and other derivative distributions. The new x32 psABI is
>>> available from:
>>>
>>> https://sites.google.com/site/x32abi/home
>>>
>>
>> Initial kernel, glibc and gcc port for x32 psABI draft version 0.2  is done:
>>
>> 1. Kernel should be very stable.  Fedora 14 kernel patch is also available.
>> 2. Need to fix GCC run-time libraries.
>> 3. Need to fix glibc tests.
>>
>> Please check them out and provide feed backs.
>
> I updated x32 website:
>
> https://sites.google.com/site/x32abi/
>
> with "get started" instructions.
>

I update the x32 website

https://sites.google.com/site/x32abi/

with data on mcf in SPEC CPU 2K:

* 181.mcf from SPEC CPU 2000:
  o Intel Core i7
+ ~32% faster than x86-64.
+ ~1% slower than ia32.
+ Sizes in bytes:

   text   databssdec
 hexfilename
   12155336   7036  19527
4c47mcf.32
   12421664  13568  26653
681dmcf.64
   11583412   7040  19035
4a5bmcf.x32

  o Intel Atom
+ ~28% faster than x86-64.
+ ~0.5% slower than ia32.
+ Sizes in bytes:

   text   databssdec
 hexfilename
12059336   7036  19431
4be7mcf.32
12341664  13568  26573
67cdmcf.64
11411412   7040  18863
49afmcf.x32


-- 
H.J.

Re: x32 psABI status update

[H.J. Lu]

On Wed, Feb 16, 2011 at 9:45 PM, H.J. Lu  wrote:
> On Wed, Feb 16, 2011 at 11:22 AM, H.J. Lu  wrote:
>> Hi,
>>
>> I updated  x32 psABI draft to version 0.2 to change x32 library path
>> from lib32 to libx32 since lib32 is used for ia32 libraries on Debian,
>> Ubuntu and other derivative distributions. The new x32 psABI is
>> available from:
>>
>> https://sites.google.com/site/x32abi/home
>>
>
> Initial kernel, glibc and gcc port for x32 psABI draft version 0.2  is done:
>
> 1. Kernel should be very stable.  Fedora 14 kernel patch is also available.
> 2. Need to fix GCC run-time libraries.
> 3. Need to fix glibc tests.
>
> Please check them out and provide feed backs.

I updated x32 website:

https://sites.google.com/site/x32abi/

with "get started" instructions.

-- 
H.J.

x32 psABI status update

[H.J. Lu]

On Wed, Feb 16, 2011 at 11:22 AM, H.J. Lu  wrote:
> Hi,
>
> I updated  x32 psABI draft to version 0.2 to change x32 library path
> from lib32 to libx32 since lib32 is used for ia32 libraries on Debian,
> Ubuntu and other derivative distributions. The new x32 psABI is
> available from:
>
> https://sites.google.com/site/x32abi/home
>

Initial kernel, glibc and gcc port for x32 psABI draft version 0.2  is done:

1. Kernel should be very stable.  Fedora 14 kernel patch is also available.
2. Need to fix GCC run-time libraries.
3. Need to fix glibc tests.

Please check them out and provide feed backs.

Thanks.


-- 
H.J.

Re: X32 psABI status

[H. Peter Anvin]

On 02/13/2011 03:39 PM, Alan Cox wrote:

a. the int $0x80 instruction is much slower than syscall.  An actual
i386 process can use the syscall instruction which is disambiguated
by the CPU based on mode, but an x32 process is in the same CPU mode
as a normal 64-bit process.


So set a flag, whoopee


That's what we're doing, functionally.


b. 64-bit arguments have to be split between two registers for the
i386 entry points, requiring user-space stubs.


Diddums. Given you've yet to explain why everyone desperately needs this
extra interface why do we care ?


All in all, the cost of an extra system call table is quite modest.


And the cost of not doing it is a gloriously wonderful zero. Yo've still
not explained the justification or what large number of apps are going to
use it.

It's a simple question - why do we care, why do we want the overhead and
the hassle, what do users get in return ?


The target applications are an embedded (closed or mostly closed) 
environment, and the question is if the performance gain is worth it. 
It is an open question at this stage and we'll see what the numbers look 
like and, if it turns out to be worthwhile, what exactly the final 
implementation will look like.


-hpa

Re: X32 psABI status

[H. Peter Anvin]

On 02/13/2011 01:33 PM, Alan Cox wrote:


Who actually needs this new extra API - whats the justification for
everyone having more crud dumping their kernels, more syscall paths
(which are one of the most security critical areas) and the like.

What are the benchmark numbers to justify this versus just using the
existing kernel interfaces ?



That's what the prototype is meant to show.

-hpa

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 3:39 PM, Alan Cox  wrote:
>> a. the int $0x80 instruction is much slower than syscall.  An actual
>>    i386 process can use the syscall instruction which is disambiguated
>>    by the CPU based on mode, but an x32 process is in the same CPU mode
>>    as a normal 64-bit process.
>
> So set a flag, whoopee
>
>> b. 64-bit arguments have to be split between two registers for the
>>    i386 entry points, requiring user-space stubs.
>
> Diddums. Given you've yet to explain why everyone desperately needs this
> extra interface why do we care ?
>
>> All in all, the cost of an extra system call table is quite modest.
>
> And the cost of not doing it is a gloriously wonderful zero. Yo've still
> not explained the justification or what large number of apps are going to
> use it.
>
> It's a simple question - why do we care, why do we want the overhead and
> the hassle, what do users get in return ?
>

The real question is if we need to use ia32.  If the answer is yes, then
x32 provides the benefit of ia32 with register extended to 64bit plus 8
more registers as well as IP relative address.

-- 
H.J.

Re: X32 psABI status

[Alan Cox]

> a. the int $0x80 instruction is much slower than syscall.  An actual
>i386 process can use the syscall instruction which is disambiguated
>by the CPU based on mode, but an x32 process is in the same CPU mode
>as a normal 64-bit process.

So set a flag, whoopee

> b. 64-bit arguments have to be split between two registers for the
>i386 entry points, requiring user-space stubs.

Diddums. Given you've yet to explain why everyone desperately needs this
extra interface why do we care ?

> All in all, the cost of an extra system call table is quite modest.

And the cost of not doing it is a gloriously wonderful zero. Yo've still
not explained the justification or what large number of apps are going to
use it.

It's a simple question - why do we care, why do we want the overhead and
the hassle, what do users get in return ?

Alan

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 2:57 PM, Arnd Bergmann  wrote:
>
>> The cost of an entire different ABI layer (supporting a new memory layout)
>> would be enormous, a.k.a. "not worth it", which is why the memory layout
>> of kernel objects needs to be compatible with i386.
>
> Right, this makes sense, you certainly can't redefine all the data
> structures.
>
> What would probably be a good idea is to compare the set of syscalls
> in X32 and asm-generic, and to either eliminate or document the
> differences. You can probably even take the asm-generic syscall numbers,
> even if you keep the i386 data structures.
>

x32 system call number is BASE + 64bit system call number.  It is
easy to keep 64bit and x32 in sync.

-- 
H.J.

Re: X32 psABI status

[Arnd Bergmann]

On Sunday 13 February 2011, H. Peter Anvin wrote:
> We prototyped using the int $0x80 system call entry point.  However,
> there are two disadvantages:
> 
> a. the int $0x80 instruction is much slower than syscall.  An actual
>i386 process can use the syscall instruction which is disambiguated
>by the CPU based on mode, but an x32 process is in the same CPU mode
>as a normal 64-bit process.

Well, you could simply change entry.S to allow syscall with high numbers
to have the same effect as int $0x80, but not introduce another table
to solve this.

> b. 64-bit arguments have to be split between two registers for the
>i386 entry points, requiring user-space stubs.

64 bit arguments are very rare, and most of those syscalls are not
performance critical, so this could be dealt with on a case-by-case
basis, possibly by introducing a new syscall number for the variant
passing a 64 bit register.

> All in all, the cost of an extra system call table is quite modest.

The memory size overhead may be small, but auditing another table
for every change could become a noticable burden (your though, not mine).

> The cost of an entire different ABI layer (supporting a new memory layout)
> would be enormous, a.k.a. "not worth it", which is why the memory layout
> of kernel objects needs to be compatible with i386.

Right, this makes sense, you certainly can't redefine all the data
structures. 

What would probably be a good idea is to compare the set of syscalls
in X32 and asm-generic, and to either eliminate or document the
differences. You can probably even take the asm-generic syscall numbers,
even if you keep the i386 data structures.

Arnd

Re: X32 psABI status

[H. Peter Anvin]

On 02/13/2011 02:28 PM, Arnd Bergmann wrote:
> On Sunday 13 February 2011, H. Peter Anvin wrote:
>> The actual idea is to use the i386 compat ABI for memory layout, but
>> with a 64-bit register convention.  That means that system calls that
>> don't make references to memory structures can simply use the 64-bit
>> system calls, otherwise we're planning to reuse the i386 compat system
>> calls, but invoke them via the syscall instruction (which requires a new
>> system call table) and to pass 64-bit arguments in single registers.
> 
> As far as I know, any task can already call both the 32 and 64 bit syscall
> entry points on x86. Is there anything you can't do just as well by
> using a combination of the two methods, without introducing a third one?

We prototyped using the int $0x80 system call entry point.  However,
there are two disadvantages:

a. the int $0x80 instruction is much slower than syscall.  An actual
   i386 process can use the syscall instruction which is disambiguated
   by the CPU based on mode, but an x32 process is in the same CPU mode
   as a normal 64-bit process.
b. 64-bit arguments have to be split between two registers for the
   i386 entry points, requiring user-space stubs.

All in all, the cost of an extra system call table is quite modest.  The
cost of an entire different ABI layer (supporting a new memory layout)
would be enormous, a.k.a. "not worth it", which is why the memory layout
of kernel objects needs to be compatible with i386.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: X32 psABI status

[Arnd Bergmann]

On Sunday 13 February 2011, H. Peter Anvin wrote:
> The actual idea is to use the i386 compat ABI for memory layout, but
> with a 64-bit register convention.  That means that system calls that
> don't make references to memory structures can simply use the 64-bit
> system calls, otherwise we're planning to reuse the i386 compat system
> calls, but invoke them via the syscall instruction (which requires a new
> system call table) and to pass 64-bit arguments in single registers.

As far as I know, any task can already call both the 32 and 64 bit syscall
entry points on x86. Is there anything you can't do just as well by
using a combination of the two methods, without introducing a third one?

Arnd

Re: X32 psABI status

[H. Peter Anvin]

On 02/13/2011 01:16 PM, H. Peter Anvin wrote:
> 
> The actual idea is to use the i386 compat ABI for memory layout, but
> with a 64-bit register convention.  That means that system calls that
> don't make references to memory structures can simply use the 64-bit
> system calls, otherwise we're planning to reuse the i386 compat system
> calls, but invoke them via the syscall instruction (which requires a new
> system call table) and to pass 64-bit arguments in single registers.
> 

Oh, and as to why not copy the i386 system call list straight off... we
don't really want to add a new ABI with crap like sys_socketcall.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 2:03 PM, H. Peter Anvin  wrote:
> On 02/13/2011 01:28 PM, H.J. Lu wrote:
>>
>> That is is currently implemented on hjl/x32 branch.
>>
>> I also added
>>
>> __NR_sigaction
>> __NR_sigpending
>> __NR_sigprocmask
>> __NR_sigsuspend
>>
>> to help the Bionic C library.
>>
>
> That seems a little redundant... even on the i386 front we want people
> to use the rt_sig* system calls.  As a porting aid I can see it, but we
> should avoid deprecated system calls in the final version.
>

That is the plan. I don't want to spend more time on fixing the
Bionic C library. Once x32 glibc is running, I will remove those
ia32 system calls.

-- 
H.J.

Re: X32 psABI status

[H. Peter Anvin]

On 02/13/2011 01:28 PM, H.J. Lu wrote:
> 
> That is is currently implemented on hjl/x32 branch.
> 
> I also added
> 
> __NR_sigaction
> __NR_sigpending
> __NR_sigprocmask
> __NR_sigsuspend
> 
> to help the Bionic C library.
> 

That seems a little redundant... even on the i386 front we want people
to use the rt_sig* system calls.  As a porting aid I can see it, but we
should avoid deprecated system calls in the final version.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: X32 psABI status

[Alan Cox]

> The actual idea is to use the i386 compat ABI for memory layout, but
> with a 64-bit register convention.  That means that system calls that
> don't make references to memory structures can simply use the 64-bit
> system calls, otherwise we're planning to reuse the i386 compat system
> calls, but invoke them via the syscall instruction (which requires a new
> system call table) and to pass 64-bit arguments in single registers.

Who actually needs this new extra API - whats the justification for
everyone having more crud dumping their kernels, more syscall paths
(which are one of the most security critical areas) and the like.

What are the benchmark numbers to justify this versus just using the
existing kernel interfaces ?

Alan

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 1:16 PM, H. Peter Anvin  wrote:
> On 02/13/2011 01:10 PM, H.J. Lu wrote:
>>> The basic concept looks entirely reasonable to me, but I'm
>>> curious what drove the decision to start out with the x86_64
>>> system calls instead of the generic ones.
>>>
>>> Since tile was merged, we now have support for compat syscalls
>>> in the generic syscall ABI. I would have assumed that it
>>> was possible to just use those if you decide to do a new
>>> ABI in the first place.
>>>
>>> The other option that would have appeared natural to me is
>>> to just use the existing 32 bit compat ABI with the few
>>> necessary changes done based on the personality.
>
> The actual idea is to use the i386 compat ABI for memory layout, but
> with a 64-bit register convention.  That means that system calls that
> don't make references to memory structures can simply use the 64-bit
> system calls, otherwise we're planning to reuse the i386 compat system
> calls, but invoke them via the syscall instruction (which requires a new
> system call table) and to pass 64-bit arguments in single registers.
>

That is is currently implemented on hjl/x32 branch.

I also added

__NR_sigaction
__NR_sigpending
__NR_sigprocmask
__NR_sigsuspend

to help the Bionic C library.


-- 
H.J.

Re: X32 psABI status

[H. Peter Anvin]

On 02/13/2011 01:10 PM, H.J. Lu wrote:
>>>
>>> 1. Kernel interface with syscall is close to be finalized.
>>

I don't think calling it "finalized" is accurate... it is more
accurately described as "prototyped".

>> Really? I haven't seen this being posted for review yet ;-)
>>
>> The basic concept looks entirely reasonable to me, but I'm
>> curious what drove the decision to start out with the x86_64
>> system calls instead of the generic ones.
>>
>> Since tile was merged, we now have support for compat syscalls
>> in the generic syscall ABI. I would have assumed that it
>> was possible to just use those if you decide to do a new
>> ABI in the first place.
>> 
>> The other option that would have appeared natural to me is
>> to just use the existing 32 bit compat ABI with the few
>> necessary changes done based on the personality.

The actual idea is to use the i386 compat ABI for memory layout, but
with a 64-bit register convention.  That means that system calls that
don't make references to memory structures can simply use the 64-bit
system calls, otherwise we're planning to reuse the i386 compat system
calls, but invoke them via the syscall instruction (which requires a new
system call table) and to pass 64-bit arguments in single registers.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 12:10 PM, Arnd Bergmann  wrote:
> On Saturday 12 February 2011 20:41:01 H.J. Lu wrote:
>> Hi,
>>
>> We made lots of progresses on x32 pABI:
>>
>> https://sites.google.com/site/x32abi/
>>
>> 1. Kernel interface with syscall is close to be finalized.
>
> Really? I haven't seen this being posted for review yet ;-)
>
> The basic concept looks entirely reasonable to me, but I'm
> curious what drove the decision to start out with the x86_64
> system calls instead of the generic ones.
>
> Since tile was merged, we now have support for compat syscalls
> in the generic syscall ABI. I would have assumed that it
> was possible to just use those if you decide to do a new
> ABI in the first place.
>
> The other option that would have appeared natural to me is
> to just use the existing 32 bit compat ABI with the few
> necessary changes done based on the personality.
>

If you are interested, please check out the x32 kernel to
take a look and provide feedback.

Thanks.

-- 
H.J.

Re: X32 psABI status

[Arnd Bergmann]

On Saturday 12 February 2011 20:41:01 H.J. Lu wrote:
> Hi,
> 
> We made lots of progresses on x32 pABI:
> 
> https://sites.google.com/site/x32abi/
> 
> 1. Kernel interface with syscall is close to be finalized.

Really? I haven't seen this being posted for review yet ;-)

The basic concept looks entirely reasonable to me, but I'm
curious what drove the decision to start out with the x86_64
system calls instead of the generic ones.

Since tile was merged, we now have support for compat syscalls
in the generic syscall ABI. I would have assumed that it
was possible to just use those if you decide to do a new
ABI in the first place.

The other option that would have appeared natural to me is
to just use the existing 32 bit compat ABI with the few
necessary changes done based on the personality.

Arnd

Re: X32 psABI status

[Joseph S. Myers]

On Sun, 13 Feb 2011, Petr Baudis wrote:

> I think it would be great if you could add some text like this plus some
> rationale (AIUI, this is geared mainly at new Atoms and other x86_64
> embedded platforms) to the document.
> 
> (BTW, it is not really convincing to me that such a niche is worth all
> the trouble this is going to bring.)

It seems to me there's a much more general utility for this.  After all, 
the normal practice on 64-bit Power Architecture GNU/Linux systems, say, 
is for most programs to be 32-bit and only a few that have a use for a 
large address space to be 64-bit.  For most programs, large pointers and 
size_t are just a waste of memory (and so of cache) - but the extra 
registers are significantly beneficial (as are such things as being able 
to assume SSE to be supported).  (Large files, 64-bit time_t, etc., 
however, are of wider use than large address space, but as I noted in 
 it would be a fairly 
substantial project to address all such issues of inappropriate arbitrary 
limits in C APIs.)

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: X32 psABI status

[H.J. Lu]

2011/2/13 Petr Baudis :
> On Sun, Feb 13, 2011 at 07:13:57AM -0800, H.J. Lu wrote:
>> On Sun, Feb 13, 2011 at 7:07 AM, Florian Weimer  wrote:
>> > * H. J. Lu:
>> >
>> >>> Actually, I'm wondering if you can do the translation in user space.
>> >>> There already are 32-on-64 implementations in existence, without
>> >>> kernel changes (recent Hotspot, LuaJIT, and probably some more).
>> >>
>> >> Please check out the x32 kernel source and provide feedback.
>> >
>> > I still don't understand why you need a separate syscall table.  You
>> > should really be able to run on an unmodified amd64 kernel, in 64 bit
>>
>> That is done on purpose. x32 is designed for environments where the
>> current ia32 API is sufficient. You can think it as ia32 with register
>> extended to 64bit plus 8 more registers. Everything else is still 32bit.
>
> I think it would be great if you could add some text like this plus some
> rationale (AIUI, this is geared mainly at new Atoms and other x86_64
> embedded platforms) to the document.

I updated:

https://sites.google.com/site/x32abi/

> (BTW, it is not really convincing to me that such a niche is worth all
> the trouble this is going to bring.)
>

That is a good question.  Otherwise x32 would have been implemented
long time ago.


-- 
H.J.

Re: X32 psABI status

[Petr Baudis]

On Sun, Feb 13, 2011 at 07:13:57AM -0800, H.J. Lu wrote:
> On Sun, Feb 13, 2011 at 7:07 AM, Florian Weimer  wrote:
> > * H. J. Lu:
> >
> >>> Actually, I'm wondering if you can do the translation in user space.
> >>> There already are 32-on-64 implementations in existence, without
> >>> kernel changes (recent Hotspot, LuaJIT, and probably some more).
> >>
> >> Please check out the x32 kernel source and provide feedback.
> >
> > I still don't understand why you need a separate syscall table.  You
> > should really be able to run on an unmodified amd64 kernel, in 64 bit
> 
> That is done on purpose. x32 is designed for environments where the
> current ia32 API is sufficient. You can think it as ia32 with register
> extended to 64bit plus 8 more registers. Everything else is still 32bit.

I think it would be great if you could add some text like this plus some
rationale (AIUI, this is geared mainly at new Atoms and other x86_64
embedded platforms) to the document.

(BTW, it is not really convincing to me that such a niche is worth all
the trouble this is going to bring.)

-- 
Petr "Pasky" Baudis
Computer science education cannot make an expert programmer any more
than studying brushes and pigment can make an expert painter. --esr

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 7:43 AM, Maciej W. Rozycki  wrote:
> On Sun, 13 Feb 2011, Florian Weimer wrote:
>
>> >> Actually, I'm wondering if you can do the translation in user space.
>> >> There already are 32-on-64 implementations in existence, without
>> >> kernel changes (recent Hotspot, LuaJIT, and probably some more).
>> >
>> > Please check out the x32 kernel source and provide feedback.
>>
>> I still don't understand why you need a separate syscall table.  You
>> should really be able to run on an unmodified amd64 kernel, in 64 bit
>> mode.  This would imply that tools like strace don't need any porting
>> at all (you could just use the amd64 version), and even GDB would
>> mostly worked unchanged.
>
>  For the record -- I suggested a similar approach for n32 MIPS too (back
> when it was on the table), but people rejected it deciding it was easier
> for them to add a separate syscall table (for a change).  It was perhaps
> even more surprising as any MIPS 32-bit user pointer is a valid 64-bit one
> too (I suspect this is also the case with x86-64) and any simple type,
> including pointers and the "long long" type (such as used with lseek64(2),
> etc.) goes into a single 64-bit register or stack slot with both ABIs, so
> any conversion layer (boundary checks or whatever; structures can be
> sorted out with padding) in libc would be pretty thin.

x32 is the same 64bit kernel interface for system calls which takes 64bit
arguments.  You can pass either 32bit or 64bit value to them.

-- 
H.J.

Re: X32 psABI status

[Maciej W. Rozycki]

On Sun, 13 Feb 2011, Florian Weimer wrote:

> >> Actually, I'm wondering if you can do the translation in user space.
> >> There already are 32-on-64 implementations in existence, without
> >> kernel changes (recent Hotspot, LuaJIT, and probably some more).
> >
> > Please check out the x32 kernel source and provide feedback.
> 
> I still don't understand why you need a separate syscall table.  You
> should really be able to run on an unmodified amd64 kernel, in 64 bit
> mode.  This would imply that tools like strace don't need any porting
> at all (you could just use the amd64 version), and even GDB would
> mostly worked unchanged.

 For the record -- I suggested a similar approach for n32 MIPS too (back 
when it was on the table), but people rejected it deciding it was easier 
for them to add a separate syscall table (for a change).  It was perhaps 
even more surprising as any MIPS 32-bit user pointer is a valid 64-bit one 
too (I suspect this is also the case with x86-64) and any simple type, 
including pointers and the "long long" type (such as used with lseek64(2), 
etc.) goes into a single 64-bit register or stack slot with both ABIs, so 
any conversion layer (boundary checks or whatever; structures can be 
sorted out with padding) in libc would be pretty thin.

 One argument in favour was the need of some people for crippled 
interfaces such as original lseek(2) that would fail on large files for 
the sake of some broken programs out there they wanted to rebuild for the 
new ABI without fixing, sigh...  Actually some OSes, such as NetBSD (I 
think, it could have been one of the other *BSDs), do not offer these 
crippled interfaces at all on any platform, but I gather people simply are 
not particularly interested into pushing portability that far.

 So now we have another table in the kernel to maintain that goes wrong in 
respect to the two others from time to time.  But there you go...  At 
least each of the three is optional.  I couldn't care less about n32 
anyway; I usually configure 64-bit MIPS kernels for n64 only.

  Maciej

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 7:21 AM, Florian Weimer  wrote:
> * H. J. Lu:
>
>> On Sun, Feb 13, 2011 at 7:07 AM, Florian Weimer  wrote:
>>> * H. J. Lu:
>>>
> Actually, I'm wondering if you can do the translation in user space.
> There already are 32-on-64 implementations in existence, without
> kernel changes (recent Hotspot, LuaJIT, and probably some more).

 Please check out the x32 kernel source and provide feedback.
>>>
>>> I still don't understand why you need a separate syscall table.  You
>>> should really be able to run on an unmodified amd64 kernel, in 64 bit
>>
>> That is done on purpose. x32 is designed for environments where the
>> current ia32 API is sufficient. You can think it as ia32 with register
>> extended to 64bit plus 8 more registers. Everything else is still 32bit.
>
> I think of it as amd64 where all the process memory happens to reside
> in the first 4 GB of address space, and pointers are stored as 32 bits
> (and you'd also reduce the size of longs because sizeof(long) !=
> sizeof(void *) will break too many programs).

That is what the processor sees in an x32 program.

> As I said, both LuaJIT and Hotspot are already using this model, with
> custom memory allocators and a user-space translation layers, so I
> still don't see what you get by changing the kernel.  LuaJIT has even
> implemented the amd64 ABI, so you can call C libraries from your
> 32-bit code.  (Note that LuaJIT uses 64-bit words to store 32-bit
> pointers with several tag bits, but it does so even on pure 32-bit
> platforms.)

They can continue to do so.

> If you want to make x32 closer to i386, I don't see the point.  Why
> would it be problematic if it was as close to i386 as, say, armel?
>

We are providing a 32bit API with 64bit registers. Current APIs support
either 32bit or 64bit.  I am not talking about pointer or long. I am talking
other types, like time_t, off_t, ..., defined by API.  Adding another API
is extremely difficult.

-- 
H.J.

Re: X32 psABI status

[Florian Weimer]

* H. J. Lu:

> On Sun, Feb 13, 2011 at 7:07 AM, Florian Weimer  wrote:
>> * H. J. Lu:
>>
 Actually, I'm wondering if you can do the translation in user space.
 There already are 32-on-64 implementations in existence, without
 kernel changes (recent Hotspot, LuaJIT, and probably some more).
>>>
>>> Please check out the x32 kernel source and provide feedback.
>>
>> I still don't understand why you need a separate syscall table.  You
>> should really be able to run on an unmodified amd64 kernel, in 64 bit
>
> That is done on purpose. x32 is designed for environments where the
> current ia32 API is sufficient. You can think it as ia32 with register
> extended to 64bit plus 8 more registers. Everything else is still 32bit.

I think of it as amd64 where all the process memory happens to reside
in the first 4 GB of address space, and pointers are stored as 32 bits
(and you'd also reduce the size of longs because sizeof(long) !=
sizeof(void *) will break too many programs).

As I said, both LuaJIT and Hotspot are already using this model, with
custom memory allocators and a user-space translation layers, so I
still don't see what you get by changing the kernel.  LuaJIT has even
implemented the amd64 ABI, so you can call C libraries from your
32-bit code.  (Note that LuaJIT uses 64-bit words to store 32-bit
pointers with several tag bits, but it does so even on pure 32-bit
platforms.)

If you want to make x32 closer to i386, I don't see the point.  Why
would it be problematic if it was as close to i386 as, say, armel?

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 7:07 AM, Florian Weimer  wrote:
> * H. J. Lu:
>
>>> Actually, I'm wondering if you can do the translation in user space.
>>> There already are 32-on-64 implementations in existence, without
>>> kernel changes (recent Hotspot, LuaJIT, and probably some more).
>>
>> Please check out the x32 kernel source and provide feedback.
>
> I still don't understand why you need a separate syscall table.  You
> should really be able to run on an unmodified amd64 kernel, in 64 bit

That is done on purpose. x32 is designed for environments where the
current ia32 API is sufficient. You can think it as ia32 with register
extended to 64bit plus 8 more registers. Everything else is still 32bit.

> mode.  This would imply that tools like strace don't need any porting
> at all (you could just use the amd64 version), and even GDB would
> mostly worked unchanged.
>

Yes, strace needs to be updated. I have ported GDB to x32.


-- 
H.J.

Re: X32 psABI status

[Florian Weimer]

* H. J. Lu:

>> Actually, I'm wondering if you can do the translation in user space.
>> There already are 32-on-64 implementations in existence, without
>> kernel changes (recent Hotspot, LuaJIT, and probably some more).
>
> Please check out the x32 kernel source and provide feedback.

I still don't understand why you need a separate syscall table.  You
should really be able to run on an unmodified amd64 kernel, in 64 bit
mode.  This would imply that tools like strace don't need any porting
at all (you could just use the amd64 version), and even GDB would
mostly worked unchanged.

Re: X32 psABI status

[H.J. Lu]

On Sun, Feb 13, 2011 at 12:48 AM, Florian Weimer  wrote:
> * H. Peter Anvin:
>
>> On 02/12/2011 01:10 PM, Florian Weimer wrote:
>>> Why is the ia32 compatiblity kernel interface used?
>>
>> Because there is no way in hell we're designing in a second
>> compatibility ABI in the kernel (and it has to be a compatibility ABI,
>> because of the pointer size difference.)
>
> Actually, I'm wondering if you can do the translation in user space.
> There already are 32-on-64 implementations in existence, without
> kernel changes (recent Hotspot, LuaJIT, and probably some more).

Please check out the x32 kernel source and provide feedback.

-- 
H.J.

Re: X32 psABI status

[H.J. Lu]

On Sat, Feb 12, 2011 at 9:47 PM, Matt Thomas  wrote:
>
> On Feb 12, 2011, at 7:02 PM, Andrew Pinski wrote:
>
>> On Sat, Feb 12, 2011 at 3:04 PM, H. Peter Anvin  wrote:
>>> On 02/12/2011 01:10 PM, Florian Weimer wrote:
 Why is the ia32 compatiblity kernel interface used?
>>>
>>> Because there is no way in hell we're designing in a second
>>> compatibility ABI in the kernel (and it has to be a compatibility ABI,
>>> because of the pointer size difference.)
>>
>> I think he is asking why not create a new ABI layer for the kernel
>> like it is done for n32 for MIPS.
>
> The kernel syscall ABI needs to be able to be pass 64-bit quantities
> in a single register (since that's what the calling ABI is capable
> of doing but I don't think the ia32 kernel interface can do)?

It works. Please check out the x32 kernel to take a  look.

> Maybe it's me, but I expected X32 to be the X86-64 ABI with 32-bit longs
> and pointers (converted to 64-bit arguments when passed in register or
> on the stack).  That allows the same syscall argument marshalling that
> currently exists but just need a different set of syscall vectors.
>

That is exactly what we implemented.

-- 
H.J.

Re: X32 psABI status

[Florian Weimer]

* H. Peter Anvin:

> On 02/12/2011 01:10 PM, Florian Weimer wrote:
>> Why is the ia32 compatiblity kernel interface used?
>
> Because there is no way in hell we're designing in a second
> compatibility ABI in the kernel (and it has to be a compatibility ABI,
> because of the pointer size difference.)

Actually, I'm wondering if you can do the translation in user space.
There already are 32-on-64 implementations in existence, without
kernel changes (recent Hotspot, LuaJIT, and probably some more).

Re: X32 psABI status

[Matt Thomas]

On Feb 12, 2011, at 7:02 PM, Andrew Pinski wrote:

> On Sat, Feb 12, 2011 at 3:04 PM, H. Peter Anvin  wrote:
>> On 02/12/2011 01:10 PM, Florian Weimer wrote:
>>> Why is the ia32 compatiblity kernel interface used?
>> 
>> Because there is no way in hell we're designing in a second
>> compatibility ABI in the kernel (and it has to be a compatibility ABI,
>> because of the pointer size difference.)
> 
> I think he is asking why not create a new ABI layer for the kernel
> like it is done for n32 for MIPS.

The kernel syscall ABI needs to be able to be pass 64-bit quantities
in a single register (since that's what the calling ABI is capable
of doing but I don't think the ia32 kernel interface can do)?

Maybe it's me, but I expected X32 to be the X86-64 ABI with 32-bit longs
and pointers (converted to 64-bit arguments when passed in register or
on the stack).  That allows the same syscall argument marshalling that
currently exists but just need a different set of syscall vectors.

Re: X32 psABI status

[Andrew Pinski]

On Sat, Feb 12, 2011 at 3:04 PM, H. Peter Anvin  wrote:
> On 02/12/2011 01:10 PM, Florian Weimer wrote:
>> Why is the ia32 compatiblity kernel interface used?
>
> Because there is no way in hell we're designing in a second
> compatibility ABI in the kernel (and it has to be a compatibility ABI,
> because of the pointer size difference.)

I think he is asking why not create a new ABI layer for the kernel
like it is done for n32 for MIPS.

-- Andrew

Re: X32 psABI status

[H. Peter Anvin]

On 02/12/2011 01:10 PM, Florian Weimer wrote:
> Why is the ia32 compatiblity kernel interface used?

Because there is no way in hell we're designing in a second
compatibility ABI in the kernel (and it has to be a compatibility ABI,
because of the pointer size difference.)

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.

Re: X32 psABI status

[Matt Thomas]

On Feb 12, 2011, at 1:29 PM, H.J. Lu wrote:

> On Sat, Feb 12, 2011 at 1:10 PM, Florian Weimer  wrote:
>> * H. J. Lu:
>> 
>>> We made lots of progresses on x32 pABI:
>>> 
>>> https://sites.google.com/site/x32abi/
>>> 
>>> 1. Kernel interface with syscall is close to be finalized.
>>> 2. GCC x32 branch is stabilizing.
>>> 3. The Bionic C library works with the syscall kernel interface.
>>> 
>>> The next major milestone will be x32 glibc port.
>> 
>> It is a bit difficult to extract useful information from these
>> resources.
> 
> That is true. Contributions are more than welcome.
> 
>> Is off_t 32 bits?  Why is the ia32 compatiblity kernel interface used?
> 
> Yes.

off_t is not part of the psABI since it's OS dependent.

>> I'm sure a lot of people want to get rid of that in cases where they
>> control the whole software stack.
> 
> That is debatable. The current thought is the x32 user space API
> is the same as is ia32.  time_t is also an issue.

Any system call method is beyond the scope of the psABI since it's
OS dependent and user-code should never care.

Re: X32 psABI status

[H.J. Lu]

On Sat, Feb 12, 2011 at 1:10 PM, Florian Weimer  wrote:
> * H. J. Lu:
>
>> We made lots of progresses on x32 pABI:
>>
>> https://sites.google.com/site/x32abi/
>>
>> 1. Kernel interface with syscall is close to be finalized.
>> 2. GCC x32 branch is stabilizing.
>> 3. The Bionic C library works with the syscall kernel interface.
>>
>> The next major milestone will be x32 glibc port.
>
> It is a bit difficult to extract useful information from these
> resources.

That is true. Contributions are more than welcome.

> Is off_t 32 bits?  Why is the ia32 compatiblity kernel interface used?

Yes.

> I'm sure a lot of people want to get rid of that in cases where they
> control the whole software stack.

That is debatable. The current thought is the x32 user space API
is the same as is ia32.  time_t is also an issue.

> Is stack alignment to 16 bytes beneficial for X32?
>

Even ia32 uses 16byte stack alignment and we will make it
official soon via an amendment to the i386 psABI.

-- 
H.J.

Re: X32 psABI status

[Florian Weimer]

* H. J. Lu:

> We made lots of progresses on x32 pABI:
>
> https://sites.google.com/site/x32abi/
>
> 1. Kernel interface with syscall is close to be finalized.
> 2. GCC x32 branch is stabilizing.
> 3. The Bionic C library works with the syscall kernel interface.
>
> The next major milestone will be x32 glibc port.

It is a bit difficult to extract useful information from these
resources.

Is off_t 32 bits?  Why is the ia32 compatiblity kernel interface used?
I'm sure a lot of people want to get rid of that in cases where they
control the whole software stack.

Is stack alignment to 16 bytes beneficial for X32?

X32 psABI status

[H.J. Lu]

Hi,

We made lots of progresses on x32 pABI:

https://sites.google.com/site/x32abi/

1. Kernel interface with syscall is close to be finalized.
2. GCC x32 branch is stabilizing.
3. The Bionic C library works with the syscall kernel interface.

The next major milestone will be x32 glibc port.

-- 
H.J.