64 bit API/ABI changes proposal for -current
About 34 system calls need work (with time changes), approximately
16 without time changes. I've included a summary at the end.
The questions before us:
* What to do about time representation.
* Whether to create a new syscall vector or use the existing
vector.
I have thought long and hard about this and I now believe that we
should keep our existing vector, even though it's become a real mess,
at least through the release. This will also allow us to convert
to a 64 bit time representation without blowing anything up. The
original API calls would of course remain valid.
wait4() -> wait464()
*stat() -> *stat64()
getrusage() -> getrusage64()
getfsstat() -> getfsstat64()
*statfs() -> *statfs64()
*itimer() -> *itimer64()
select()-> select64()
gettimeofday() -> gettimeofday64()
settimeofday() -> settimeofday64()
*times()-> *times64()
adjtime() -> adjtime64()
quotactl() -> quotactl64()
CREATE sleep64() (takes timeval64)
typedef int64_t time64_t;
struct timeval64 {
time64_ttv_sec;
int64_t tv_frac;/* N/2^63 fractional */
};
typedef struct timeval64 timespec64;
struct itimerval64 {
struct timeval64 it_interval;
struct timeval64 it_value;
int64_t it_resolution; /* N/2^63 fractional (ro) */
}
#define TIMEFRAC0x4000LL
I understand some people may want this to be unsigned and
0xF[63] or 0x7F[62] but this makes the fractional value
an imperfect representation in both base 10 and base 2 and its
a good idea to make it a perfect representation in at least base 10
or base 2. Also, divider logic is far faster with fewer '1' bits.
In terms of signed verses unsigned it is far easier to use a signed
value here in regards to program logic and useage and, additionally,
being able to represent negative times is a useful abstraction even
if the syscalls do not particularly need the capability.
In regards to normalizing the API such that a call to stat() should
actually be a call to stat64(), I believe a compiler option may be
the best way to go. The compiler option would simply pre-set a
#define and our includes would #define-rename the functions to
present a normalized API to those programs that can handle it.
Example:
gcc --unix64(sets __UNIX_API64__)
gcc --unix32(unsets __UNIX_API64__) (default)
Then our system include files would do this (in appropriate places):
#ifdef __UNIX_API64__
#define statstat64
#define ... (etc)
#define time_t time64_t
#define timeval timeval64
#endif
Amoung other things this should hopefully make any issues that come
up debuggable by causing the compiler to generate warnings when
prototypes do not match up.
EVENTUAL GOAL
The eventual goal would then be to compile our entire source tree
with --unix64 through a release or two, and then, eventually (perhaps
two years down the line) we would make --unix64 the default.
TIMEVAL64 ISSUES
Now there is a fairly serious issue with using fractional seconds, and
that is that you cannot use the representation additively. That is,
you cannot represent '100ns' in the structure and then add thousands
of structures together and get the result you expect because 100ns
cannot be represented exactly with this method.
This is the same problem that financial systems have when they try to
use floating point (e.g. 'double') to add monetary amounts together.
If you do not round each intermediate result to the required resolution
errors can creep into long calculations. As long as people understand
this problem I believe the format is reasonable.
-Matt
Matthew Dillon
<[EMAIL PROTECTED]>
(Matt's summary of syscall changes)
wait4 extend rusage elements to 64 bits
getrusage extend rusage elements to 64 bits
getfsstat extend f_blocks, f_bfree, f_bavail, f_files,
f_ffree, f_syncwrites, f_asyncwrites,
f_syncreads, f_asyncreads to 64 bits,
add any additional fields required by
Kirk.
*statfs various internal fields and time
*stat ino_t -> 64 bits, time fields. What about st_gen ?
*itimer itimerval contains timeval (also, add
Re: 64 bit API/ABI changes proposal for -current
In message <[EMAIL PROTECTED]>, Matthew Dillon w
rites:
> struct timeval64 {
> time64_ttv_sec;
> int64_t tv_frac;/* N/2^63 fractional */
> };
We have this one already, and it's called bintime, except that it
correctly uses N/2^64 fractional the way binary computers prefer it.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
:> struct timeval64 {
:> time64_ttv_sec;
:> int64_t tv_frac;/* N/2^63 fractional */
:> };
:
:We have this one already, and it's called bintime, except that it
:correctly uses N/2^64 fractional the way binary computers prefer it.
:
:--
:Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
Hmm. That's certainly a reasonable point. I suppose a negative
representation is still possible if one considers the entire 128
bit word as a 128 bit fractional time.
All right, I'll amend the proposal to use 2^64. the fractional
element will be unsigned, the tv_sec will remain signed.
-Matt
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
In message <[EMAIL PROTECTED]>, Matthew Dillon w
rites:
>
>:> struct timeval64 {
>:> time64_ttv_sec;
>:> int64_t tv_frac;/* N/2^63 fractional */
>:> };
>:
>:We have this one already, and it's called bintime, except that it
>:correctly uses N/2^64 fractional the way binary computers prefer it.
>:
>:--
>:Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
>
>Hmm. That's certainly a reasonable point. I suppose a negative
>representation is still possible if one considers the entire 128
>bit word as a 128 bit fractional time.
>
>All right, I'll amend the proposal to use 2^64. the fractional
>element will be unsigned, the tv_sec will remain signed.
That is exactly how bintime is defined :-)
struct bintime {
time_t sec;
uint64_t frac;
};
If I had a int128_t, I would have used that instead...
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
:>
:>All right, I'll amend the proposal to use 2^64. the fractional
:>element will be unsigned, the tv_sec will remain signed.
:
:That is exactly how bintime is defined :-)
:
: struct bintime {
: time_t sec;
: uint64_t frac;
: };
:
:If I had a int128_t, I would have used that instead...
:
:
:--
:Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
Ok, we have an issue in regards to libc/user function visibility.
The bintime structures and functions are surrounded by __BSD_VISIBLE.
The question to you and to the list in general is: what to call the
user-visible structure. 'bintime' is a cute name, I certainly like
it better then timeval64, and we could probably get away with calling
the user visible structure bintime, but I don't know if we can get
away with including all the supporting inline functions (not that we
necessarily have to include them for the syscall work, but it would
be nice).
Also, the in-kernel time_t is 32 bits on 32 bit architectures so bintime
is not compatible as-is, but it would not be much work to make
bintime use time64_t. We can't create yet another userland time
structure without making seconds 64 bits.
-Matt
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
In message <[EMAIL PROTECTED]>, Matthew Dillon w rites: >Ok, we have an issue in regards to libc/user function visibility. >The bintime structures and functions are surrounded by __BSD_VISIBLE. That was an attempt at preventive debrucification :-) >The question to you and to the list in general is: what to call the >user-visible structure. 'bintime' is a cute name, I certainly like >it better then timeval64, and we could probably get away with calling >the user visible structure bintime, but I don't know if we can get >away with including all the supporting inline functions (not that we >necessarily have to include them for the syscall work, but it would >be nice). One of the thing which irks me about timeval and timespec is that the perfectly useable and practically needed functions for adding, subtracting and normalizing them are not allowed in the .h in POSIX. POSIX sucks in many cases like that: It defines just enough to cramp future improvement, but not enough to facilitate current usability :-( When I first encountered POSIXMISTAKE many years ago I thought it kind of rude, now I'm tempted to invert the sign and introduce "POSIXGOTTHISRIGHT", it would be far less intrusive. The right thing to do is probably to introduce which is pulled into and possibly unless people insist on getting pure-to-the-heart-POSIX-includes. >Also, the in-kernel time_t is 32 bits on 32 bit architectures so bintime >is not compatible as-is, but it would not be much work to make >bintime use time64_t. We can't create yet another userland time >structure without making seconds 64 bits. It was my expectation that time_t would become 64 bits in the kernel, rather than have to s/time_t/time64_t throughout the kernel. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
Matthew Dillon wrote:
> About 34 system calls need work (with time changes), approximately
> 16 without time changes. I've included a summary at the end.
>
> The questions before us:
>
> * What to do about time representation.
>
> * Whether to create a new syscall vector or use the existing
> vector.
>
> I have thought long and hard about this and I now believe that we
> should keep our existing vector, even though it's become a real mess,
> at least through the release. This will also allow us to convert
> to a 64 bit time representation without blowing anything up. The
> original API calls would of course remain valid.
>
> wait4() -> wait464()
> *stat() -> *stat64()
> getrusage() -> getrusage64()
> getfsstat() -> getfsstat64()
> *statfs() -> *statfs64()
> *itimer() -> *itimer64()
> select()-> select64()
> gettimeofday() -> gettimeofday64()
> settimeofday() -> settimeofday64()
> *times()-> *times64()
> adjtime() -> adjtime64()
> quotactl() -> quotactl64()
Matt, I realize you have thought long and hard about this, but I want to
make a couple of points.
1) 32 bit i386 already has its fate sealed. AMD are switching to 64 bit,
and Intel are doing the same (either x86-64 or their own 64 bit environment,
depending on which rumors you listen to).
2) Many of these calls *are* 64 bit already on 64 bit platforms. Adding
an additional layer here just makes life harder when we 64 bit is the norm.
For example, get/settimeofday, *itimer, select, etc etc use timeval:
struct timeval {
longtv_sec; /* seconds */
longtv_usec;/* and microseconds */
};
and that, folks, is 64 bit on 64 bit platforms already.
3) 5.0 is not going to last till anywhere near year 2038. In the next year,
or two at tops, it is going to be perfectly clear what the future of common
64 bit hardware will look like
4) Changing time_t to 64 bit using long long on a native 32 bit platform
is a f*cking nightmare. I've tried this and it got very nasty very quickly.
5) Changing time_t to 64 bit is trivial on a native 64 bit platform. I
have done this on an ia64 machine. The sparc64 folks indicated they wanted
to do the same. Much of the work has been done already.
I worry that jumping in prematurely will leave us with a legacy issues that
will haunt us for year and years to come, while the linux folks laugh at
our pain and convoluted API's because they've already set their internal
time_t to 64 bit on their 64 bit platforms.
I urge folks to hold off just a bit longer so that we can get 5.x stable
and in shape and released without blowing up 50% of our ports due to long
long time_t problems. re@ is already having nightmares about 5.0, we do
not need this thrown into the mix yet. Especially when the right solution
will become more obvious as the 64 bit desktop machine dust settles.
Also, hackers@ is really not the place for this.
Cheers,
-Peter
--
Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
"All of this is for nothing if we don't go to the stars" - JMS/B5
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
:Matt, I realize you have thought long and hard about this, but I want to
:make a couple of points.
:
:1) 32 bit i386 already has its fate sealed. AMD are switching to 64 bit,
:and Intel are doing the same (either x86-64 or their own 64 bit environment,
:depending on which rumors you listen to).
This is wishful thinking, which I will elaborate on below.
:2) Many of these calls *are* 64 bit already on 64 bit platforms. Adding
:an additional layer here just makes life harder when we 64 bit is the norm.
:For example, get/settimeofday, *itimer, select, etc etc use timeval:
:struct timeval {
:longtv_sec; /* seconds */
:longtv_usec;/* and microseconds */
:};
:and that, folks, is 64 bit on 64 bit platforms already.
:
:3) 5.0 is not going to last till anywhere near year 2038. In the next year,
:or two at tops, it is going to be perfectly clear what the future of common
:64 bit hardware will look like
Ok, now wait a second. First of all our time_t problems do not start
in 2038. THEY ARE ALREADY HERE. A 32 bit time_t is an unusable
representation for a growing number of applications. Simulations,
forward looking contract time/date representation, databases, hell
even my USENET news server uses a 64 bit time representation internally.
Second, by far the most important reason for giving 32 bit platforms
a 64 bit time capability is software portability, and that is an issue
NOW. People who develop software on 32 bit platforms do not have
universal access to 64 bit platforms in order to ensure that their
code works. Every single application I've written that uses time_t
heavily has required modifications when ported to a 64 bit platform,
even though I tried to take into account 64 bit time_t's in the
development of the software.
Being able to write an application that uses 64 bit time_t's on a 32
bit platform now is therefore an important goal.
Third, you have a very short term view of the longevity of software.
Software I wrote 10 years ago is likely to still be in operation in
2038. One of the new products I am working on uses PC/104 technology
and some of those units are almost guarenteed to still be in operation
in 2037. When 2037 comes rolling around I'm not likely to want to pull
out the "old source" and try to "make it work".
This is a problem for me *NOW*, not in 2038. But NOW.
Forth, FreeBSD is used heavily in the embedded world and the intention
seems to be to make it even more useable. A good chunk of the embedded
world will almost certainly still be using 32 bit platforms in 2038.
64 bits is great, but 64 bits eats a great deal of electrical power.
This is why you still see little 8 bit cpus in the embedded world
today. 32 bit platforms are likely to be in major use in 2038.
:4) Changing time_t to 64 bit using long long on a native 32 bit platform
:is a f*cking nightmare. I've tried this and it got very nasty very quickly.
This is not what I proposed. I proposed adding new system calls to
completely and permanently fix all of our outstanding issues and then
providing a compiler option to allow developers to select which API
they want presented.
I'm not sure what the correct solution is but I am sure that we
need one, and that we should do at least the kernel bits to support
a solution for the 5.0 release. But I will tell you something... I
am not a fan of basic system data types changing sizes whether you
happen to be running your software on a 32 bit platform or a 64 bit
platform. Our off_t works extremely well simply because it's big
enough and because it is the same across all platforms. Until we
are able to supply kernel support for similar functionality with other
data types - time specifications, inodes, block numbers, and so forth,
our API and ABI will remain broken in my view.
:5) Changing time_t to 64 bit is trivial on a native 64 bit platform. I
:have done this on an ia64 machine. The sparc64 folks indicated they wanted
:to do the same. Much of the work has been done already.
:
:I worry that jumping in prematurely will leave us with a legacy issues that
:will haunt us for year and years to come, while the linux folks laugh at
:our pain and convoluted API's because they've already set their internal
:time_t to 64 bit on their 64 bit platforms.
:
:I urge folks to hold off just a bit longer so that we can get 5.x stable
:and in shape and released without blowing up 50% of our ports due to long
:long time_t problems. re@ is already having nightmares about 5.0, we do
:not need this thrown into the mix yet. Especially when the right solution
:will become more obvious as the 64 bit desktop machine dust settles.
Again, nothing I proposed would have any effect on our ports. All of
my proposals to date are designed to avoid screwing up -current.
Re: 64 bit API/ABI changes proposal for -current
My original proposal, before this one, was to create a separate ABI for all the new calls, which also means creating a duplicate set of libraries. I'm still game to do that -- it could be controlled by a make.conf variable and selectable via a compiler option. If we maintain timeval and timespec (except for the 64 bit time_t) then we have full portability. The only real work required inside the kernel is to make the kernel internal time representation 64 bits unconditionally, which is not a big deal, and to implement the syscall abstraction that was proposed in the "stack gap" thread by Ian Dowse. Then the ABI works becomes far easier. This work is mostly just rearranging existing code a little, not implementing new algorithms, and I don't see how it could possibly break -current. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
Matthew Dillon wrote:
> :4) Changing time_t to 64 bit using long long on a native 32 bit platform
> :is a f*cking nightmare. I've tried this and it got very nasty very quickly.
>
> This is not what I proposed. I proposed adding new system calls to
> completely and permanently fix all of our outstanding issues and then
> providing a compiler option to allow developers to select which API
> they want presented.
Well, one thing that I would not be against is a clean divorce of the
syscall layer and libc. That then gives us the freedom to implement
alternative API selections etc at compile time pretty easily.
I just really do not want to see this sort of thing turning up:
time_t foo = time(0);
printf("the time is %lld\n", (long long)foo); /* i386 compatability */
.. because that hurts our 64 bit platforms down the road when long long
becomes 128 bit. intmax_t/%j etc has similar problems as it then takes
the native 64 bit time type (long) and converts it to a synthetic 128 bit
type. ie: %j and intmax_t etc are just as bad.
My first exposure to a unix OS had some pretty horrific stat structure
versioning and evilness (svr4). It was a real nightmare, except you never
got to wake up and discover that it was all over.
I am not against providing 64 bit extentions for 32 bit systems, but it
really needs to be done so that it all becomes a giant NOP when on native
64 bit systems. Something like the large-file-summit API extensions that
all magically go away after the transition period or when you get a chance
to do a total ABI breakaway.
Cheers,
-Peter
--
Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
"All of this is for nothing if we don't go to the stars" - JMS/B5
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
:Well, one thing that I would not be against is a clean divorce of the
:syscall layer and libc. That then gives us the freedom to implement
:alternative API selections etc at compile time pretty easily.
:
:I just really do not want to see this sort of thing turning up:
:
: time_t foo = time(0);
: printf("the time is %lld\n", (long long)foo); /* i386 compatability */
:
:.. because that hurts our 64 bit platforms down the road when long long
:becomes 128 bit. intmax_t/%j etc has similar problems as it then takes
:the native 64 bit time type (long) and converts it to a synthetic 128 bit
:type. ie: %j and intmax_t etc are just as bad.
:
:My first exposure to a unix OS had some pretty horrific stat structure
:versioning and evilness (svr4). It was a real nightmare, except you never
:got to wake up and discover that it was all over.
:
:I am not against providing 64 bit extentions for 32 bit systems, but it
:really needs to be done so that it all becomes a giant NOP when on native
:64 bit systems. Something like the large-file-summit API extensions that
:all magically go away after the transition period or when you get a chance
:to do a total ABI breakaway.
:
:Cheers,
:-Peter
:--
:Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
Well, then what we want is a new syscall vector, duplicate libraries,
and a compiler option, and leave all the function names the same
(which means no bintime but allows us to retain everything else).
-current would release supporting both with the compiler option
defaulting to --unix32 on the IA32 and --unix64 on 64 bit platforms,
and then down the line the compiler option would default to --unix64
on all platforms, and then down the line a little more the original
syscall vector would become a compatibility option that most people
leave out.
If we do a clean cut of the ABI for the alpha, sparc, etc... for
things like ino_t, which I think we can get away with, then this all
becomes a big NOP on 64 bit platforms and eventually becomes a NOP
(only one library set and syscall vector is generated) on 32 bit
platforms. If we include the other structural changes... 64 bit
ino_t in *stat() for example, in only the --unix64 ABI, then
we encourage (but do not require) developers to migrate to the ABI.
printf("the time is %lld\n"...) is a problem that will *NEVER*
go away short of defining a format character to represent time_t
itself. The assumption that time_t is somehow tied to a 'long' is
broken, pure and simple. I actually think %j is workable, because
I don't know a single case of 'speed critical code' where you have a
*printf() call passing a time_t. It's very rare, and at least now
GCC will catch type size mismatches.
The advantage of all of this is that the 32 bit code stays intact and
compatible in both source and binary forms, which means no further
disruption of -current, just a lot of ABI work on my part. I already
did a dry run of the libc changes necessary and given the chance to
adjust things incrementally (which can be done with this methodology),
it's a lot of grunt work but nothing that I couldn't do in a few weeks.
-Matt
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
Oops, let me clarify what I mean by 'duplicate libraries'. I do *NOT* mean duping the source code, I simply mean duping the compile run in the buildworld, one for --unix32, one for --unix64, for 32 bit platforms. 64 bit platforms would require no library duplication at all (that being the idea), nor will 32 bit platforms down the line when the old 32 bit interface becomes a compatibility option. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
Matthew Dillon wrote:
> Well, then what we want is a new syscall vector, duplicate libraries,
> and a compiler option, and leave all the function names the same
> (which means no bintime but allows us to retain everything else).
> -current would release supporting both with the compiler option
> defaulting to --unix32 on the IA32 and --unix64 on 64 bit platforms,
> and then down the line the compiler option would default to --unix64
> on all platforms, and then down the line a little more the original
> syscall vector would become a compatibility option that most people
> leave out.
Argh! This is way overkill. It is *not* what I was advocating.
The time to do this is when we bump libc's major number, not duplicating
libraries etc. That's a worst case outcome especially when it isn't needed.
It would be far less disruptive to add an additional group of 64 bit
time aware API's that are optional and portable. Leave the standards-defined
API's alone.
For example, in your previous suggestion, you had:
typedef int64_t time64_t;
struct timeval64 {
time64_ttv_sec;
int64_t tv_frac;/* N/2^63 fractional */
};
#ifdef __UNIX_API64__
#define timeval timeval64
#endif
You simply *cannot do this* and remain posix compliant on 64 bit machines.
We do not just go and change posix api's just for the sake of leaving out a
"64" in a name. If you are going to modify code to use tv_frac, then you
may as well refer to it in a 'struct timeval64' and be done with it.
I suspect there would be far less confusion if they were not named so closely
to their standardized counterparts.
Changing code to work with --unix64 on 32 bit systems becomes useless
when we have 64 bit native systems, because we then need to un-port it..
(remember --unix64 changes struct timeval.tv_usec to .tv_frac, so code that
is "cleaned" to work on --unix64 will no longer compile on a native
64 bit posix compliant environment)
Maybe I'm getting all upset about nothing, but you are scaring the hell
out of me so far.
> The advantage of all of this is that the 32 bit code stays intact and
> compatible in both source and binary forms, which means no further
> disruption of -current, just a lot of ABI work on my part. I already
> did a dry run of the libc changes necessary and given the chance to
> adjust things incrementally (which can be done with this methodology),
> it's a lot of grunt work but nothing that I couldn't do in a few weeks.
Well, I'd like to see a working prototype before anything is committed.
Changes in this area are of massive importance to the whole project, so we
*must* get it right and agreed apon, or leave it alone. It would also be
really good if you could get some buy-in from the other *bsd/linux folks.
Cheers,
-Peter
--
Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
"All of this is for nothing if we don't go to the stars" - JMS/B5
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
Peter, you are again not reading what I'm posting carefully enough.
I said very specifically that we wouldn't be using the fractional
stuff for this case.
Additionally, you are making the assumption that --unix64 and --unix32
are so different from each other that the same code cannot compile to
both targets. This is not true either. The only difference is that
a couple of types, such as time_t and ino_t, would be extended to
64 bits. The same source will be able to compile to both targets.
(in fact, our existing source can almost do this already).
Perhaps I am putting too much information out here. The idea of this
discussion is to DISCUSS, I am simply creating a starting point for
the discussion. None of this stuff is set in stone so screaming at me
telling me the whole enchilada will not work because of a few things
in here is not going to be productive. Instead you should simply
advocate that the interfering items be removed to make the rest work,
or something like that.
I'm also not sure what you mean when you say "overkill". What you
seem to be proposing is an incompatible change to the API and ABI.
Sure we can fix all the code in /usr/src, but ports are another matter.
What I am proposing does not initially interfere (or need to interfere
) with ports.
Finally, in regards to a 'working prototype', please note that what
I am proposing is designed to be an incremental process precisely
because the whole enchilada is too big to create a 'working prototype'
before committing. The project can be cut into many small incremental,
non destructive pieces which are comitted incrementally. That's the
whole point. You and others may like to do wholely working prototypes
in P4 and then smash the whole thing into -current but that isn't how
I work. So, no, I am not going to go about it that way.
-Matt
Matthew Dillon
<[EMAIL PROTECTED]>
:Matthew Dillon wrote:
:
:> Well, then what we want is a new syscall vector, duplicate libraries,
:> and a compiler option, and leave all the function names the same
:> (which means no bintime but allows us to retain everything else).
:> -current would release supporting both with the compiler option
:> defaulting to --unix32 on the IA32 and --unix64 on 64 bit platforms,
:> and then down the line the compiler option would default to --unix64
:> on all platforms, and then down the line a little more the original
:> syscall vector would become a compatibility option that most people
:> leave out.
:
:Argh! This is way overkill. It is *not* what I was advocating.
:
:The time to do this is when we bump libc's major number, not duplicating
:libraries etc. That's a worst case outcome especially when it isn't needed.
:
:It would be far less disruptive to add an additional group of 64 bit
:time aware API's that are optional and portable. Leave the standards-defined
:API's alone.
:
:For example, in your previous suggestion, you had:
: typedef int64_t time64_t;
: struct timeval64 {
: time64_ttv_sec;
: int64_t tv_frac;/* N/2^63 fractional */
: };
: #ifdef __UNIX_API64__
: #define timeval timeval64
: #endif
:
:You simply *cannot do this* and remain posix compliant on 64 bit machines.
:We do not just go and change posix api's just for the sake of leaving out a
:"64" in a name. If you are going to modify code to use tv_frac, then you
:may as well refer to it in a 'struct timeval64' and be done with it.
:I suspect there would be far less confusion if they were not named so closely
:to their standardized counterparts.
:
:Changing code to work with --unix64 on 32 bit systems becomes useless
:when we have 64 bit native systems, because we then need to un-port it..
:(remember --unix64 changes struct timeval.tv_usec to .tv_frac, so code that
:is "cleaned" to work on --unix64 will no longer compile on a native
:64 bit posix compliant environment)
:
:Maybe I'm getting all upset about nothing, but you are scaring the hell
:out of me so far.
:
:> The advantage of all of this is that the 32 bit code stays intact and
:> compatible in both source and binary forms, which means no further
:> disruption of -current, just a lot of ABI work on my part. I already
:> did a dry run of the libc changes necessary and given the chance to
:> adjust things incrementally (which can be done with this methodology),
:> it's a lot of grunt work but nothing that I couldn't do in a few weeks.
:
:Well, I'd like to see a working prototype before anything is committed.
:Changes in this area are of massive importance to the whole project, so we
:*must* get it right and agreed apon, or leave it alone. It would also be
:really good
Re: 64 bit API/ABI changes proposal for -current
Matthew Dillon wrote: > Ok, we have an issue in regards to libc/user function visibility. > The bintime structures and functions are surrounded by __BSD_VISIBLE. > > The question to you and to the list in general is: what to call the > user-visible structure. 'bintime' is a cute name, I certainly like > it better then timeval64, and we could probably get away with calling > the user visible structure bintime, but I don't know if we can get > away with including all the supporting inline functions (not that we > necessarily have to include them for the syscall work, but it would > be nice). What do Linux, Solaris, SCO, etc. call it? Whatever it is, it should be the same, so that source code does not need to be "#if"'ed. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
Peter Wemm wrote:
> Matthew Dillon wrote:
> > :4) Changing time_t to 64 bit using long long on a native 32 bit platform
> > :is a f*cking nightmare. I've tried this and it got very nasty very quickly.
> >
> > This is not what I proposed. I proposed adding new system calls to
> > completely and permanently fix all of our outstanding issues and then
> > providing a compiler option to allow developers to select which API
> > they want presented.
>
> Well, one thing that I would not be against is a clean divorce of the
> syscall layer and libc. That then gives us the freedom to implement
> alternative API selections etc at compile time pretty easily.
I was really, really tempted to suggest this, but I couldn't
come up with a way to make it work as a shared interface under
a single libc, without doing the fixup in the crt0 ala the
dlopen fixup for use of code in the ld.so (and have different
crt0/ld.so's). The 32 bit versions could be stubbed to the 64's
in the stub code, right there, which would make the system calls
DTRT on both platforms.
The other downside appears to be that the only way to do this
entirely cleanly is to give up on the static linking support,
so that the lib-linked-with-lib works per the ELF specification,
which is a real bikeshed. 8-(.
> I just really do not want to see this sort of thing turning up:
>
> time_t foo = time(0);
> printf("the time is %lld\n", (long long)foo); /* i386 compatability */
>
> .. because that hurts our 64 bit platforms down the road when long long
> becomes 128 bit. intmax_t/%j etc has similar problems as it then takes
> the native 64 bit time type (long) and converts it to a synthetic 128 bit
> type. ie: %j and intmax_t etc are just as bad.
I think the issue comes down to running 32 bit code on 64 bit
platforms, particularly, Intel code, and having it "just work".
I don't know how to avoid the problem, without adding sizing
to the types (e.g. "time64_t").
This is a very tempting thing to do; that changes your code
into:
time64_t foo = time64(0);
printf("the time is %lld\n", foo);
Or:
time_t foo = time(0);
printf("the time is %ld\n", foo); /* WARNING */
I think that the argument comes down to internal use of 64 bit
types on 32 bit platforms, with the 64/32 conversion done at
the system call interface.
PHK's right: POSIX sucks.
> My first exposure to a unix OS had some pretty horrific stat structure
> versioning and evilness (svr4). It was a real nightmare, except you never
> got to wake up and discover that it was all over.
>
> I am not against providing 64 bit extentions for 32 bit systems, but it
> really needs to be done so that it all becomes a giant NOP when on native
> 64 bit systems. Something like the large-file-summit API extensions that
> all magically go away after the transition period or when you get a chance
> to do a total ABI breakaway.
The real problem in the example is printf; it's because there's
a sized integer type specifier in the format string, instead of
an unsized one, and the compiler whines about this. That's what
will end up making the code different on a 32 bit machine, no
matter what. 8-(.
Maybe in this case, GCC has gotten too smart for it's own good?
IMO, for a long time now, POSIX has belonged in a "libposix",
which is built on top of the native system services, but that's
*really* a lot to ask...
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
Matthew Dillon wrote: > Well, then what we want is a new syscall vector, duplicate libraries, > and a compiler option, and leave all the function names the same > (which means no bintime but allows us to retain everything else). > -current would release supporting both with the compiler option > defaulting to --unix32 on the IA32 and --unix64 on 64 bit platforms, > and then down the line the compiler option would default to --unix64 > on all platforms, and then down the line a little more the original > syscall vector would become a compatibility option that most people > leave out. Not to be a party-pooper, but I think this will fail as soon as you have a program that wants to link against a third party library that calls entry points in libc. It doesn't even have to be a binary-only thing; think of trying to build packages for the purposes of a CDROM distribution. You would end up needing to do the same "--unix32/--unix64" thing for every library you build. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
: :Matthew Dillon wrote: :> Well, then what we want is a new syscall vector, duplicate libraries, :> and a compiler option, and leave all the function names the same :> (which means no bintime but allows us to retain everything else). :> -current would release supporting both with the compiler option :> defaulting to --unix32 on the IA32 and --unix64 on 64 bit platforms, :> and then down the line the compiler option would default to --unix64 :> on all platforms, and then down the line a little more the original :> syscall vector would become a compatibility option that most people :> leave out. : : :Not to be a party-pooper, but I think this will fail as soon :as you have a program that wants to link against a third party :library that calls entry points in libc. : :It doesn't even have to be a binary-only thing; think of trying :to build packages for the purposes of a CDROM distribution. You :would end up needing to do the same "--unix32/--unix64" thing for :every library you build. : :-- Terry Well I figured one would just use the default for third party libraries. The idea is to get the system sources working first as well as support developers who need the feature, and worry about ports and so forth later. Ports might not be able to use 64 bit functionality under 32 bit OSs until it becomes the default under 32 bit OSs. The moment we even *attempt* to introduce changes to things ports and third part libraries use, we blow up half the ports tree. Oh, wait, we've ALREADY blown up half the ports tree. On nearly every release! Its one of the big complaints I hear about FreeBSD. -Matt Matthew Dillon <[EMAIL PROTECTED]> To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
In message: <[EMAIL PROTECTED]> Matthew Dillon <[EMAIL PROTECTED]> writes: : Peter, you are again not reading what I'm posting carefully enough. : I said very specifically that we wouldn't be using the fractional : stuff for this case. I think that this is a worthy problem to solve. But not for 5.0. It is too late in the game to be playing games this big with the ABI. It will have to wait for 6.0, even if the problem is extremely pressing and hurting people today. It is just too late. We should be focusing on polishing/finishing the features that are already in the tree, or nearly ready to go into the tree. The scope of this proposal seems too large given that we're two or three months out from a release. If you are looking ahead to 6.0, then I think now is a good time to start since we can do the cut over just after we do the branch in a few months. This will also give us time to prototype and test things so we don't have any false starts. We may even be able to retrofit it into 5.1 or 5.2, but even if we can't that's OK. I mean, I've stalled for three years getting cardbus into a release. If we can live w/o a clear and present technology for a couple of years, we certainly can wait a little bit to solve the time problem. imho. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
In message <[EMAIL PROTECTED]>, "M. Warner Losh" writes: >In message: <[EMAIL PROTECTED]> >I think that this is a worthy problem to solve. But not for 5.0. [...] Good thinking. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: 64 bit API/ABI changes proposal for -current
At the risk of butting into an important thread and revealing my total ignorance about all this stuff, I notice that NetBSD claims to be "64 bit clean" since "1.0", right down to FFS (I looked around the NetBSD mailing lists recently for discussions about UFS2--there didn't appear to be much interest in their camp for it, kind of odd I thought) . So does that mean they had to make big changes to their system call interface/ABI? If so, would their experiences/approach be of any use to FreeBSD? Andrew Lankford To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
