Re: svn commit: r266553 - head/release/scripts
On Tue, 27 May 2014 14:31:44 -0600 Warner Losh wrote: On May 27, 2014, at 1:40 PM, Tijl Coosemans t...@freebsd.org wrote: On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: On May 27, 2014, at 1:36 AM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: So I?m still waiting for a use case that requires the new names. One has not been articulated, and I don?t think one actually exists. Imagine you've built a system with MACHINE_ARCH amd64 and one with MACHINE_ARCH mips64. Now you want to populate these systems with a list of packages for which you wrote a script. These systems each support 2 ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one (x32 and mipsn32). Both are native in the sense that they make full use of the instruction set. This is not like i386 on amd64 or mips o32 on mips64 because those are more like compat shims that operate under special (crippled) cpu modes that nobody uses unless they're stuck with old code. Both our ABIs on the other hand are native and equally valid and which one to use for a particular package depends entirely on the use case. If your use case requires more than 4G of address space you'll have to use the 64-bit package, otherwise you can use the 32-bit package which, depending on how pointer heavy the code is, may give a performance benefit. You make this choice for each of the packages in your list and add that information to your script. Long hypothetical, but so what? In such a case, you’d pick one of two different MACHINE_ARCH values depending on the package. This this is a fairly atypical use case, it would not be unreasonable for the person wanting to do this to know the proper companion ABI. In both cases, you have the choice of two other values to use. Which one you use will depend on a variety of factors, and what might be right for one application may be wrong for others. And in both cases, there’s actually two choices: for amd64, you’d have i386 and x32. Both of these are fine choices, and it would depend on the workload which one is better (i386 has better toolchain support and maturity, x32 offers some interesting theoretical wins, but doesn’t have the same maturity). Same with mips64, you’d have two choices as well. In both cases, you can’t just take uname/MACHINE_ARCH and slap :32 on the end. As far as I can see nobody uses i386 on amd64 except in cases where you know you're dealing with i386 like an i386 jail, or you need to run an old i386 binary that you're stuck with, or with compat shims like Wine. The choice of i386 is not comparable with the generic choice between ILP32 or LP64 on a 64-bit instruction set, which is why I'm ok with changing the x86 in the pkg scheme to either i386 or amd64 like you are proposing. If pkg would keep using instruction set families like x86 or mips then you could distinguish between old 32-bit, new 32-bit and new 64-bit using 3 generic suffixes to avoid lookup tables. The pattern of an old 32-bit instruction set that got extended to 64-bit on which you can define a new 32-bit ABI is common to many instruction set families, but I'm ok with old 32-bit and new 64-bit having separate MACHINE_ARCH values. Now let's work with Nathan's patch which uses the following string to identify the pkg repository to fetch packages from: `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` On our two systems that would be FreeBSD:11:amd64 and FreeBSD:11:mips64. Now if your script has to install the 32-bit version of a package how can it go from those two strings to FreeBSD:11:x32 or FreeBSD:11:mipsn32 without a lookup table? You couldn’t. Which is the whole reason I want to have them have a standard name so you don’t need the lookup table for the common case. This is an “off in the weeds” case, and optimizing for it doesn’t make sense. Especially because in each case, you have two different 32-bit ABIs to choose from. You’d have to have some kind of table in either case. Also, the proper name for n32, in your current system, is mips:32:n32, which (a) is wrong and (b) isn’t regular. Well, it may not make sense to you and be off in the weeds, but it does to me. Very few processes actually need more than 4G of virtual address space. I cannot immediately find anything in /bin or /usr/bin for instance (maybe clang when compiling a very large C++ file?). I'm not sure where you're getting mips:32:n32 from. We're talking in the context of Nathan's patch here. I say, you can more easily indicate whether you want the 32-bit or 64-bit package by appending :32 or :64 to the original strings, so FreeBSD:11:amd64:32 and FreeBSD:11:mips64:32. Except there’s no such thing as mips64:32 in the current system. There’s two different ABIs that could mean. It could be o32 or n32, with the same sort of trade offs. There’d need to be a person in the loop to know, so there’s already a need to have special knowledge. You
Re: svn commit: r266553 - head/release/scripts
On May 28, 2014, at 4:50 AM, Tijl Coosemans t...@freebsd.org wrote: On Tue, 27 May 2014 14:31:44 -0600 Warner Losh wrote: On May 27, 2014, at 1:40 PM, Tijl Coosemans t...@freebsd.org wrote: On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: On May 27, 2014, at 1:36 AM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: So I?m still waiting for a use case that requires the new names. One has not been articulated, and I don?t think one actually exists. Imagine you've built a system with MACHINE_ARCH amd64 and one with MACHINE_ARCH mips64. Now you want to populate these systems with a list of packages for which you wrote a script. These systems each support 2 ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one (x32 and mipsn32). Both are native in the sense that they make full use of the instruction set. This is not like i386 on amd64 or mips o32 on mips64 because those are more like compat shims that operate under special (crippled) cpu modes that nobody uses unless they're stuck with old code. Both our ABIs on the other hand are native and equally valid and which one to use for a particular package depends entirely on the use case. If your use case requires more than 4G of address space you'll have to use the 64-bit package, otherwise you can use the 32-bit package which, depending on how pointer heavy the code is, may give a performance benefit. You make this choice for each of the packages in your list and add that information to your script. Long hypothetical, but so what? In such a case, you’d pick one of two different MACHINE_ARCH values depending on the package. This this is a fairly atypical use case, it would not be unreasonable for the person wanting to do this to know the proper companion ABI. In both cases, you have the choice of two other values to use. Which one you use will depend on a variety of factors, and what might be right for one application may be wrong for others. And in both cases, there’s actually two choices: for amd64, you’d have i386 and x32. Both of these are fine choices, and it would depend on the workload which one is better (i386 has better toolchain support and maturity, x32 offers some interesting theoretical wins, but doesn’t have the same maturity). Same with mips64, you’d have two choices as well. In both cases, you can’t just take uname/MACHINE_ARCH and slap :32 on the end. As far as I can see nobody uses i386 on amd64 except in cases where you know you're dealing with i386 like an i386 jail, or you need to run an old i386 binary that you're stuck with, or with compat shims like Wine. The choice of i386 is not comparable with the generic choice between ILP32 or LP64 on a 64-bit instruction set, which is why I'm ok with changing the x86 in the pkg scheme to either i386 or amd64 like you are proposing. If pkg would keep using instruction set families like x86 or mips then you could distinguish between old 32-bit, new 32-bit and new 64-bit using 3 generic suffixes to avoid lookup tables. The pattern of an old 32-bit instruction set that got extended to 64-bit on which you can define a new 32-bit ABI is common to many instruction set families, but I'm ok with old 32-bit and new 64-bit having separate MACHINE_ARCH values. OK. So we use MACHINE_ARCH for the pkg scheme. I’m glad we agree on this point. That will make adding support for pkg repos and multiple machine builds to nanobsd much easier. Now let's work with Nathan's patch which uses the following string to identify the pkg repository to fetch packages from: `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` On our two systems that would be FreeBSD:11:amd64 and FreeBSD:11:mips64. Now if your script has to install the 32-bit version of a package how can it go from those two strings to FreeBSD:11:x32 or FreeBSD:11:mipsn32 without a lookup table? You couldn’t. Which is the whole reason I want to have them have a standard name so you don’t need the lookup table for the common case. This is an “off in the weeds” case, and optimizing for it doesn’t make sense. Especially because in each case, you have two different 32-bit ABIs to choose from. You’d have to have some kind of table in either case. Also, the proper name for n32, in your current system, is mips:32:n32, which (a) is wrong and (b) isn’t regular. Well, it may not make sense to you and be off in the weeds, but it does to me. Very few processes actually need more than 4G of virtual address space. I cannot immediately find anything in /bin or /usr/bin for instance (maybe clang when compiling a very large C++ file?). I'm not sure where you're getting mips:32:n32 from. We're talking in the context of Nathan's patch here. In the context of Nathan’s patch, mips:32:n32 is the “old” name that is an alias for mipsn32. There’s no “mips64:32” defined at all, so that pattern breaks. That was my specific point. I
Re: svn commit: r266553 - head/release/scripts
On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. pgpeySv5p9K4i.pgp Description: PGP signature
Re: svn commit: r266553 - head/release/scripts
On May 28, 2014, at 9:28 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. I doubt it. The MACHINE_ARCH is used to select which files to build. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r266553 - head/release/scripts
On Wed, May 28, 2014 at 09:35:27AM -0600, Warner Losh wrote: On May 28, 2014, at 9:28 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. I doubt it. The MACHINE_ARCH is used to select which files to build. Do I understand you right that the comment references e.g. a selection of arch-specific subdir in lib/libc or libexec/rtld-elf for inclusion into the build ? If yes, I cannot disagree with the statement. My note was about our build system which currently requires full-fledged cross-build to even create i386 binary on amd64 vs. other builds which consider this as a (often minor) variations of the host target. Sure, some variances must be allowed, e.g. to select proper .S file for the ABI, but we do not need cross-build to get i386 on amd64. pgpByBEJWf8mu.pgp Description: PGP signature
Re: svn commit: r266553 - head/release/scripts
On 05/28/14 08:47, Konstantin Belousov wrote: On Wed, May 28, 2014 at 09:35:27AM -0600, Warner Losh wrote: On May 28, 2014, at 9:28 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. I doubt it. The MACHINE_ARCH is used to select which files to build. Do I understand you right that the comment references e.g. a selection of arch-specific subdir in lib/libc or libexec/rtld-elf for inclusion into the build ? If yes, I cannot disagree with the statement. My note was about our build system which currently requires full-fledged cross-build to even create i386 binary on amd64 vs. other builds which consider this as a (often minor) variations of the host target. Sure, some variances must be allowed, e.g. to select proper .S file for the ABI, but we do not need cross-build to get i386 on amd64. Does it actually do that? Full cross-build implies it builds a new compiler, at least to me, but it just builds with -m32 and changes MACHINE_ARCH. In any case, I think we have gotten lots of issues crossed here. I'll try to separate them out: 1. We will not ever have mixed-and-matched x32 and amd64 packages on one system without a massive overhaul of ports, since you can't link x32 binaries against amd64 libraries and vice versa. Handling that requires more dependency tracking and, at a minimum, something like /usr/local/lib-$MACHINE_ARCH. Maybe that's worth doing -- I don't know -- but it is very different from -fPIC and is an orthogonal discussion. 2. PowerPC is, as you note, arguably the most integrated 32/64-bit port we have, since it was originally specified as a 64-bit instruction set. 32-bit PowerPC is much more analogous to something like x32 than to i386. And it still has a different MACHINE_ARCH, needs a different C library and RTLD (and not through #ifdef!), etc. 3. The whole discussion was originally about what we call packages. Do we use the same arbitray names that the base system and ports use, or do we use different arbitrary names? That I think has been resolved. I suspect the actual patches got lost in this very long thread, so I'll forward them to freebsd-ports@ shortly. -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 28, 2014, at 9:47 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 09:35:27AM -0600, Warner Losh wrote: On May 28, 2014, at 9:28 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. I doubt it. The MACHINE_ARCH is used to select which files to build. Do I understand you right that the comment references e.g. a selection of arch-specific subdir in lib/libc or libexec/rtld-elf for inclusion into the build ? If yes, I cannot disagree with the statement. As far as I can tell, that’s the only reason we’re doing it.. But it is a critically important reason... My note was about our build system which currently requires full-fledged cross-build to even create i386 binary on amd64 vs. other builds which consider this as a (often minor) variations of the host target. Sure, some variances must be allowed, e.g. to select proper .S file for the ABI, but we do not need cross-build to get i386 on amd64. lib32 uses -m32 and some other flags to achieve its ends. So it doesn’t create a full i386 compiler, etc. It just uses the amd64 one with special flags/args. So I don’t think it requires a full-fledged cross-build environment, or I misunderstand what you mean by that phrase. But none of this changes the fact that we have a unique MACHINE_ARCH value per ABI. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r266553 - head/release/scripts
On Wed, May 28, 2014 at 01:53:28PM -0600, Warner Losh wrote: On May 28, 2014, at 9:47 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 09:35:27AM -0600, Warner Losh wrote: On May 28, 2014, at 9:28 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. I doubt it. The MACHINE_ARCH is used to select which files to build. Do I understand you right that the comment references e.g. a selection of arch-specific subdir in lib/libc or libexec/rtld-elf for inclusion into the build ? If yes, I cannot disagree with the statement. As far as I can tell, that?s the only reason we?re doing it.. But it is a critically important reason... My note was about our build system which currently requires full-fledged cross-build to even create i386 binary on amd64 vs. other builds which consider this as a (often minor) variations of the host target. Sure, some variances must be allowed, e.g. to select proper .S file for the ABI, but we do not need cross-build to get i386 on amd64. lib32 uses -m32 and some other flags to achieve its ends. So it doesn?t create a full i386 compiler, etc. It just uses the amd64 one with special flags/args. So I don?t think it requires a full-fledged cross-build environment, or I misunderstand what you mean by that phrase. We install the headers for the MACHINE_ARCH into the compat32 build tree, and use them instead of the target MACHINE headers. The fact that toolchain -m32 works as the cross-compiler just saves the build time. But none of this changes the fact that we have a unique MACHINE_ARCH value per ABI. Warner pgpBW3gQ44lGv.pgp Description: PGP signature
Re: svn commit: r266553 - head/release/scripts
On 05/28/14 18:53, Konstantin Belousov wrote: On Wed, May 28, 2014 at 01:53:28PM -0600, Warner Losh wrote: On May 28, 2014, at 9:47 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 09:35:27AM -0600, Warner Losh wrote: On May 28, 2014, at 9:28 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. I doubt it. The MACHINE_ARCH is used to select which files to build. Do I understand you right that the comment references e.g. a selection of arch-specific subdir in lib/libc or libexec/rtld-elf for inclusion into the build ? If yes, I cannot disagree with the statement. As far as I can tell, that?s the only reason we?re doing it.. But it is a critically important reason... My note was about our build system which currently requires full-fledged cross-build to even create i386 binary on amd64 vs. other builds which consider this as a (often minor) variations of the host target. Sure, some variances must be allowed, e.g. to select proper .S file for the ABI, but we do not need cross-build to get i386 on amd64. lib32 uses -m32 and some other flags to achieve its ends. So it doesn?t create a full i386 compiler, etc. It just uses the amd64 one with special flags/args. So I don?t think it requires a full-fledged cross-build environment, or I misunderstand what you mean by that phrase. We install the headers for the MACHINE_ARCH into the compat32 build tree, and use them instead of the target MACHINE headers. The fact that toolchain -m32 works as the cross-compiler just saves the build time. Ah, OK. That's probably pointless most of the time now. It was always pointless on PowerPC and MIPS, since they share the same headers anyway, and is probably also now unnecessary on x86 now that regular -m32 works. I don't think ia64 builds lib32 (which would require a real cross-compiler). -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 28, 2014, at 11:15 PM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/28/14 18:53, Konstantin Belousov wrote: On Wed, May 28, 2014 at 01:53:28PM -0600, Warner Losh wrote: On May 28, 2014, at 9:47 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 09:35:27AM -0600, Warner Losh wrote: On May 28, 2014, at 9:28 AM, Konstantin Belousov kostik...@gmail.com wrote: On Wed, May 28, 2014 at 08:26:58AM -0600, Warner Losh wrote: Then we disagree on this point. However, the disagreement here is kinda foundational: to build a set of libraries or sys root, you have to have a MACHINE_ARCH to make it work. Even in our current system, we set MACHINE_ARCH to i386 or powerpc when building the 32-bit binaries (note: we don?t do this for mips). This means that if we do grow x32 support, we?ll need to grow a MACHINE_ARCH for it. That?s my point: all ABIs have MACHINE_ARCH associated with them, and those are the names users are used to specifying, and are the ones that are the most natural for script writers to use. With nathan?s patches, we?re to the point where those are used, though there?s also the option of using the non-standard names if you want (e.g. amd64:32 instead of x32). I am not sure if this comment would add anything to the discussion, but other build systems do not require MACHINE_ARCH. In our terms, other build systems are happy to build: i386 binary when MACHINE is amd64 and CFLAGS contains -m32; x32 binary when MACHINE is amd64 and CFLAGS contains -mx32. For HEAD and stable/10 we finally reached the point where -m32 works, on amd64; it worked on powerpc64 from inception, AFAIU Nathan. At least this is true for dependencies limited to the base system, and not to the ports (the later is since ports do not know about multiarch). It is limitation of our build that we require MACHINE_ARCH to build other natively supported ABI binary on the host. Ideally, the hacks that treat lib32 build as the cross-compilation would go away eventually. I doubt it. The MACHINE_ARCH is used to select which files to build. Do I understand you right that the comment references e.g. a selection of arch-specific subdir in lib/libc or libexec/rtld-elf for inclusion into the build ? If yes, I cannot disagree with the statement. As far as I can tell, that?s the only reason we?re doing it.. But it is a critically important reason... My note was about our build system which currently requires full-fledged cross-build to even create i386 binary on amd64 vs. other builds which consider this as a (often minor) variations of the host target. Sure, some variances must be allowed, e.g. to select proper .S file for the ABI, but we do not need cross-build to get i386 on amd64. lib32 uses -m32 and some other flags to achieve its ends. So it doesn?t create a full i386 compiler, etc. It just uses the amd64 one with special flags/args. So I don?t think it requires a full-fledged cross-build environment, or I misunderstand what you mean by that phrase. We install the headers for the MACHINE_ARCH into the compat32 build tree, and use them instead of the target MACHINE headers. The fact that toolchain -m32 works as the cross-compiler just saves the build time. Ah, OK. That's probably pointless most of the time now. It was always pointless on PowerPC and MIPS, since they share the same headers anyway, and is probably also now unnecessary on x86 now that regular -m32 works. I don't think ia64 builds lib32 (which would require a real cross-compiler). Yea, this is an area that can likely be cleaned up from the original gross hacks that were necessary before -m32 actually worked. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r266553 - head/release/scripts
On Mon, 26 May 2014 18:47:34 -0700 Nathan Whitehorn wrote: I've written two patches today. The first (http://people.freebsd.org/~nwhitehorn/pkg_machinearch.diff) is to pkg itself and the second (http://people.freebsd.org/~nwhitehorn/pkg_bootstrap_machinearch.diff) is to the pkg bootstrapper in base. These switch pkg from using identifiers like freebsd:11:arm:be:eabi:softfp to identifiers like FreeBSD:11:armeb, matching the canonical FreeBSD platform identifiers. The strings it uses can be predicted easily from scripts, as they are identical in all cases to the output of `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p`. If those are the current names for arm then I also prefer uname -p for the third field. If each arch gets a default value of 32 or 64 for the fourth field it can also be left out in many cases. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: On May 26, 2014, at 4:18 PM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: On May 26, 2014, at 8:39 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/26/14 02:35, Tijl Coosemans wrote: I suppose you could replace the x86 in the pkg scheme with i386/amd64, but then you'd still be talking about i386:32, amd64:32 and amd64:64 instead of x86:32, x86:x32 and x86:64. I suppose you could replace these by “i386”, “x32” (or “amd64x32”) and “amd64” respectively. So you're on an amd64 or mips64 system (as indicated by uname) but you want to use the 32-bit package if possible. How does your script know about the magic x32, amd64x32 or mipsn32 strings? Wouldn't it be easier if you could just use `uname -p`:32”? Oh give me a break. You know it because you know you are building for mipsn32 because that’s what you’ve set MACHINE_ARCH or TARGET_ARCH to, No, MACHINE_ARCH or TARGET_ARCH is amd64 or mips64. You are building the 64-bit OS and then decide separately per package whether you want the ILP32 one or the LP64 one. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 27, 2014, at 1:36 AM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: On May 26, 2014, at 4:18 PM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: On May 26, 2014, at 8:39 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/26/14 02:35, Tijl Coosemans wrote: I suppose you could replace the x86 in the pkg scheme with i386/amd64, but then you'd still be talking about i386:32, amd64:32 and amd64:64 instead of x86:32, x86:x32 and x86:64. I suppose you could replace these by “i386”, “x32” (or “amd64x32”) and “amd64” respectively. So you're on an amd64 or mips64 system (as indicated by uname) but you want to use the 32-bit package if possible. How does your script know about the magic x32, amd64x32 or mipsn32 strings? Wouldn't it be easier if you could just use `uname -p`:32”? Oh give me a break. You know it because you know you are building for mipsn32 because that’s what you’ve set MACHINE_ARCH or TARGET_ARCH to, No, MACHINE_ARCH or TARGET_ARCH is amd64 or mips64. You are building the 64-bit OS and then decide separately per package whether you want the ILP32 one or the LP64 one. I think I understand why we’re talking past each other. This bit is wrong. The LP64 one has a MACHINE_ARCH of “amd64” or “mips64”. The ILP32 one will have MACHINE_ARCH of “i386” or “mips”. The weird ones (ILP32 with 64-bit registers) will have a different MACHINE_ARCH of “x32” or “mipsn32”. Selection can be done on a case by case basis, but this will be validated against the supported_abis sysctl. There’s no need to have different names here, the current standard ones do just fine, are completely sufficient and all inclusive. This is rather by definition, and your understanding of the definition sounds flawed. While we have limited support for building 32-bit binaries, it is to build 32-bit binaries for a different MACHINE_ARCH. -m32 on amd64 creates i386 binaries, not amd64:32 binaries. -m32 on powerpc64 creates powerpc binaries, not powerpc64:32 binaries. On mips, -mabi-n32 (I think the option is) is required to create the mipsn32 binaries. In every single one of these cases, there exists a MACHINE_ARCH that completely describes the binary. So I’m still waiting for a use case that requires the new names. One has not been articulated, and I don’t think one actually exists. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r266553 - head/release/scripts
On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: On May 27, 2014, at 1:36 AM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: On May 26, 2014, at 4:18 PM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: On May 26, 2014, at 8:39 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/26/14 02:35, Tijl Coosemans wrote: I suppose you could replace the x86 in the pkg scheme with i386/amd64, but then you'd still be talking about i386:32, amd64:32 and amd64:64 instead of x86:32, x86:x32 and x86:64. I suppose you could replace these by ?i386?, ?x32? (or ?amd64x32?) and ?amd64? respectively. So you're on an amd64 or mips64 system (as indicated by uname) but you want to use the 32-bit package if possible. How does your script know about the magic x32, amd64x32 or mipsn32 strings? Wouldn't it be easier if you could just use `uname -p`:32?? Oh give me a break. You know it because you know you are building for mipsn32 because that?s what you?ve set MACHINE_ARCH or TARGET_ARCH to, No, MACHINE_ARCH or TARGET_ARCH is amd64 or mips64. You are building the 64-bit OS and then decide separately per package whether you want the ILP32 one or the LP64 one. I think I understand why we?re talking past each other. This bit is wrong. The LP64 one has a MACHINE_ARCH of ?amd64? or ?mips64?. The ILP32 one will have MACHINE_ARCH of ?i386? or ?mips?. The weird ones (ILP32 with 64-bit registers) will have a different MACHINE_ARCH of ?x32? or ?mipsn32?. Selection can be done on a case by case basis, but this will be validated against the supported_abis sysctl. There?s no need to have different names here, the current standard ones do just fine, are completely sufficient and all inclusive. This is rather by definition, and your understanding of the definition sounds flawed. While we have limited support for building 32-bit binaries, it is to build 32-bit binaries for a different MACHINE_ARCH. -m32 on amd64 creates i386 binaries, not amd64:32 binaries. -m32 on powerpc64 creates powerpc binaries, not powerpc64:32 binaries. On mips, -mabi-n32 (I think the option is) is required to create the mipsn32 binaries. In every single one of these cases, there exists a MACHINE_ARCH that completely describes the binary. amd64:32 is provided by the -mx32 compiler flag. So I?m still waiting for a use case that requires the new names. One has not been articulated, and I don?t think one actually exists. Imagine you've built a system with MACHINE_ARCH amd64 and one with MACHINE_ARCH mips64. Now you want to populate these systems with a list of packages for which you wrote a script. These systems each support 2 ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one (x32 and mipsn32). Both are native in the sense that they make full use of the instruction set. This is not like i386 on amd64 or mips o32 on mips64 because those are more like compat shims that operate under special (crippled) cpu modes that nobody uses unless they're stuck with old code. Both our ABIs on the other hand are native and equally valid and which one to use for a particular package depends entirely on the use case. If your use case requires more than 4G of address space you'll have to use the 64-bit package, otherwise you can use the 32-bit package which, depending on how pointer heavy the code is, may give a performance benefit. You make this choice for each of the packages in your list and add that information to your script. Now let's work with Nathan's patch which uses the following string to identify the pkg repository to fetch packages from: `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` On our two systems that would be FreeBSD:11:amd64 and FreeBSD:11:mips64. Now if your script has to install the 32-bit version of a package how can it go from those two strings to FreeBSD:11:x32 or FreeBSD:11:mipsn32 without a lookup table? I say, you can more easily indicate whether you want the 32-bit or 64-bit package by appending :32 or :64 to the original strings, so FreeBSD:11:amd64:32 and FreeBSD:11:mips64:32. Like I said in a reply to Nathan's patch, pkg could default to :32 or :64 for every arch so it can be left out in many cases. FreeBSD:11:i386 would then be equivalent to FreeBSD:11:i386:32, FreeBSD:11:amd64 to FreeBSD:11:amd64:64, etc. This also does not preclude the existence of a mipsn32 MACHINE_ARCH in case you want to build a pure mipsn32 system (including the kernel), but I wonder how many people would use that if it is possible to have a mips64 system run n32 binaries. I don't think many people would run a pure x32 system (with x32 kernel) so I don't see the need to have an x32 value for MACHINE_ARCH (or TARGET_ARCH). ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to
Re: svn commit: r266553 - head/release/scripts
On May 27, 2014, at 1:40 PM, Tijl Coosemans t...@freebsd.org wrote: On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: On May 27, 2014, at 1:36 AM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: On May 26, 2014, at 4:18 PM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: On May 26, 2014, at 8:39 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/26/14 02:35, Tijl Coosemans wrote: I suppose you could replace the x86 in the pkg scheme with i386/amd64, but then you'd still be talking about i386:32, amd64:32 and amd64:64 instead of x86:32, x86:x32 and x86:64. I suppose you could replace these by ?i386?, ?x32? (or ?amd64x32?) and ?amd64? respectively. So you're on an amd64 or mips64 system (as indicated by uname) but you want to use the 32-bit package if possible. How does your script know about the magic x32, amd64x32 or mipsn32 strings? Wouldn't it be easier if you could just use `uname -p`:32?? Oh give me a break. You know it because you know you are building for mipsn32 because that?s what you?ve set MACHINE_ARCH or TARGET_ARCH to, No, MACHINE_ARCH or TARGET_ARCH is amd64 or mips64. You are building the 64-bit OS and then decide separately per package whether you want the ILP32 one or the LP64 one. I think I understand why we?re talking past each other. This bit is wrong. The LP64 one has a MACHINE_ARCH of ?amd64? or ?mips64?. The ILP32 one will have MACHINE_ARCH of ?i386? or ?mips?. The weird ones (ILP32 with 64-bit registers) will have a different MACHINE_ARCH of ?x32? or ?mipsn32?. Selection can be done on a case by case basis, but this will be validated against the supported_abis sysctl. There?s no need to have different names here, the current standard ones do just fine, are completely sufficient and all inclusive. This is rather by definition, and your understanding of the definition sounds flawed. While we have limited support for building 32-bit binaries, it is to build 32-bit binaries for a different MACHINE_ARCH. -m32 on amd64 creates i386 binaries, not amd64:32 binaries. -m32 on powerpc64 creates powerpc binaries, not powerpc64:32 binaries. On mips, -mabi-n32 (I think the option is) is required to create the mipsn32 binaries. In every single one of these cases, there exists a MACHINE_ARCH that completely describes the binary. amd64:32 is provided by the -mx32 compiler flag. True, but not relevant. So I?m still waiting for a use case that requires the new names. One has not been articulated, and I don?t think one actually exists. Imagine you've built a system with MACHINE_ARCH amd64 and one with MACHINE_ARCH mips64. Now you want to populate these systems with a list of packages for which you wrote a script. These systems each support 2 ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one (x32 and mipsn32). Both are native in the sense that they make full use of the instruction set. This is not like i386 on amd64 or mips o32 on mips64 because those are more like compat shims that operate under special (crippled) cpu modes that nobody uses unless they're stuck with old code. Both our ABIs on the other hand are native and equally valid and which one to use for a particular package depends entirely on the use case. If your use case requires more than 4G of address space you'll have to use the 64-bit package, otherwise you can use the 32-bit package which, depending on how pointer heavy the code is, may give a performance benefit. You make this choice for each of the packages in your list and add that information to your script. Long hypothetical, but so what? In such a case, you’d pick one of two different MACHINE_ARCH values depending on the package. This this is a fairly atypical use case, it would not be unreasonable for the person wanting to do this to know the proper companion ABI. In both cases, you have the choice of two other values to use. Which one you use will depend on a variety of factors, and what might be right for one application may be wrong for others. And in both cases, there’s actually two choices: for amd64, you’d have i386 and x32. Both of these are fine choices, and it would depend on the workload which one is better (i386 has better toolchain support and maturity, x32 offers some interesting theoretical wins, but doesn’t have the same maturity). Same with mips64, you’d have two choices as well. In both cases, you can’t just take uname/MACHINE_ARCH and slap :32 on the end. Now let's work with Nathan's patch which uses the following string to identify the pkg repository to fetch packages from: `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` On our two systems that would be FreeBSD:11:amd64 and FreeBSD:11:mips64. Now if your script has to install the 32-bit version of a package how can it go from those two strings to FreeBSD:11:x32 or FreeBSD:11:mipsn32
Re: svn commit: r266553 - head/release/scripts
On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: On May 24, 2014, at 5:53 PM, Warner Losh i...@bsdimp.com wrote: On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that’s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can install packages for on this machine, which is what you really want to know. So I’m having trouble connecting the dots between this and what you are saying here. I still am absolutely flabbergasted why the MACHINE_ARCH names aren’t necessary and sufficient for packaging. I’ve yet to see any coherent reason to not use them. Why do I care that they match? Good question. When I was doing FreeNAS, I looked at integrating pkgng into nanobsd. At the time this was quite difficult because every single architecture name was different between pkgng and MACHINE_ARCH. This would mean I’d have to drag around a huge table to know how to translate one to the other (there was no simple regex either, and things like mipsn32 wouldn’t have fit into the scheme at the time). I would very much like us to see us keep these names in sync and avoid large translation tables that are difficult to maintain. Now, do you need to get it from uname -p? No. If you want to parse elf files to get it, that’s fine, so long as the names map directly to the MACHINE_ARCH names that we’ve been using for years. They completely describe the universe of supported platforms. Are they perfect? No, around the edge there may be an odd-ball that’s possible to build, but is unsupported and likely doesn’t work at all. Have we learned from these mistakes? Yes. Anything that’s actively supported has a proper name. This name is needed, btw, so that any machine can self-host, a nice feature of the /usr/src system. ABI consists of the following elements: - OS - OS ABI version (major version number in FreeBSD) - instruction set - programming model (ILP32 or LP64) - byte order (little/big endian) These are almost orthogonal dimensions in the sense that almost any combination is possible. (A combination that isn't possible is a 32-bit instruction set with LP64.) What you are asking for now is to combine two dimensions into one and combination in this case means multiplication so if you have 3 instruction sets and 2 programming models, the combined dimension needs 6 different values. You need to make the case for why you think this is a good idea. For the past 20 years we got away with this because on every installation of FreeBSD we only used one programming model at a time. This is still the case for byte order of course. What I'm saying is to keep the option open for installations with multiple programming models, where most binaries could use ILP32 and only the ones that actually need a 64-bit address space use LP64. You query the instruction set using uname and the programming models using getconf. I suppose you could replace the x86 in the pkg scheme with i386/amd64, but then you'd still be talking about i386:32, amd64:32 and amd64:64 instead of x86:32, x86:x32 and x86:64. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On 05/26/14 02:35, Tijl Coosemans wrote: On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: On May 24, 2014, at 5:53 PM, Warner Losh i...@bsdimp.com wrote: On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that’s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can install packages for on this machine, which is what you really want to know. So I’m having trouble connecting the dots between this and what you are saying here. I still am absolutely flabbergasted why the MACHINE_ARCH names aren’t necessary and sufficient for packaging. I’ve yet to see any coherent reason to not use them. Why do I care that they match? Good question. When I was doing FreeNAS, I looked at integrating pkgng into nanobsd. At the time this was quite difficult because every single architecture name was different between pkgng and MACHINE_ARCH. This would mean I’d have to drag around a huge table to know how to translate one to the other (there was no simple regex either, and things like mipsn32 wouldn’t have fit into the scheme at the time). I would very much like us to see us keep these names in sync and avoid large translation tables that are difficult to maintain. Now, do you need to get it from uname -p? No. If you want to parse elf files to get it, that’s fine, so long as the names map directly to the MACHINE_ARCH names that we’ve been using for years. They completely describe the universe of supported platforms. Are they perfect? No, around the edge there may be an odd-ball that’s possible to build, but is unsupported and likely doesn’t work at all. Have we learned from these mistakes? Yes. Anything that’s actively supported has a proper name. This name is needed, btw, so that any machine can self-host, a nice feature of the /usr/src system. ABI consists of the following elements: - OS - OS ABI version (major version number in FreeBSD) - instruction set - programming model (ILP32 or LP64) - byte order (little/big endian) These are almost orthogonal dimensions in the sense that almost any combination is possible. (A combination that isn't possible is a 32-bit instruction set with LP64.) What you are asking for now is to combine two dimensions into one and combination in this case means multiplication so if you have 3 instruction sets and 2 programming models, the combined dimension needs 6 different values. You need to make the case for why you think this is a good idea. For the past 20 years we got away with this because on every installation of FreeBSD we only used one programming model at a time. This is still the case for byte order of course. What I'm saying is to keep the option open for installations with multiple programming models, where most binaries could use ILP32 and only the ones that actually need a 64-bit address space use LP64. You query the instruction set using uname and the programming models using getconf. I suppose you could replace the x86 in the pkg scheme with i386/amd64, but then you'd still be talking about i386:32, amd64:32 and amd64:64 instead of x86:32, x86:x32 and x86:64. No. We support multiple models now and have for ten years. That's what MACHINE_ARCH is for: it defines the choice of the last three things you list above. Specifically, a shared value of MACHINE_ARCH guarantees and OS version guarantees, in FreeBSD-land, complete binary compatibility of executables. Kernels support multiple ones, in general (e.g. i386 binaries on amd64, powerpc binaries on powerpc64). They may support more in the future (x32 on amd64, potentially even cross-endian binaries). We have a nice flexible scheme in FreeBSD for supporting this. If you want to find out the list of the things the installed kernel can run, check the kern.supported_archs sysctl. Simple. These strings are just as expressive as the ones in pkg. They are the standard. They're what external build systems test against, what the src, doc, and ports trees use to define what to do universally. It's what users and code expect. The wheel we've had for 20 years is perfectly good -- why invent a new, incompatible one? -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 26, 2014, at 8:39 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/26/14 02:35, Tijl Coosemans wrote: On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: On May 24, 2014, at 5:53 PM, Warner Losh i...@bsdimp.com wrote: On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that’s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can install packages for on this machine, which is what you really want to know. So I’m having trouble connecting the dots between this and what you are saying here. I still am absolutely flabbergasted why the MACHINE_ARCH names aren’t necessary and sufficient for packaging. I’ve yet to see any coherent reason to not use them. Why do I care that they match? Good question. When I was doing FreeNAS, I looked at integrating pkgng into nanobsd. At the time this was quite difficult because every single architecture name was different between pkgng and MACHINE_ARCH. This would mean I’d have to drag around a huge table to know how to translate one to the other (there was no simple regex either, and things like mipsn32 wouldn’t have fit into the scheme at the time). I would very much like us to see us keep these names in sync and avoid large translation tables that are difficult to maintain. Now, do you need to get it from uname -p? No. If you want to parse elf files to get it, that’s fine, so long as the names map directly to the MACHINE_ARCH names that we’ve been using for years. They completely describe the universe of supported platforms. Are they perfect? No, around the edge there may be an odd-ball that’s possible to build, but is unsupported and likely doesn’t work at all. Have we learned from these mistakes? Yes. Anything that’s actively supported has a proper name. This name is needed, btw, so that any machine can self-host, a nice feature of the /usr/src system. ABI consists of the following elements: - OS - OS ABI version (major version number in FreeBSD) These two are encoded in FreeBSD and major version. There’s no problem encoding these in the package architecture string. They are easily scriptable and totally obvious to FreeBSD users and pose no problems. Nobody is opposed to these, and actually they are rather a good idea. - instruction set - programming model (ILP32 or LP64) - byte order (little/big endian) These three are encoded in MACHINE_ARCH and have been for quite some time. And you forgot several things as well: register conventions, calling conventions, stack alignment, struct alignment, pointer conversion conventions, address space layout, page size constraints, etc. There are simply far too many to try to break down like you are trying to do. And that’s even before we get into shared library conventions... These are almost orthogonal dimensions in the sense that almost any combination is possible. (A combination that isn't possible is a 32-bit instruction set with LP64.) All of these items are encoded in MACHINE_ARACH and have been for at least a decade. There’s no new argument here. If they were actually orthogonal, then that would be one thing. But they aren’t. They are all closely interrelated and we only support a vanishingly small number of possible conventions. Combinatorically, it can be hundreds. Practically, it is usually only a handful. What you are asking for now is to combine two dimensions into one and combination in this case means multiplication so if you have 3 instruction sets and 2 programming models, the combined dimension needs 6 different values. You need to make the case for why you think this is a good idea. Because uanme has to be 6 different things so the right binaries are built. It is really that simple. And we’ve already made the case, and have been using this convention for a very long time. It works. I’m not sure that the burden is on us to justify why a convention that’s been in use since FreeBSD 6 needs to not change. As weird as you might think it is, it is a convention that our users understand. For the past 20 years we got away with this because on every installation of FreeBSD we only used one programming model at a time. This is still the case for byte order of course. This isn’t true. For the past 15 years we’ve supported two programming models on amd64 at the same time. For longer than that we’ve supported linux emulation on i386. The project has known
Re: svn commit: r266553 - head/release/scripts
On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: On May 26, 2014, at 8:39 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/26/14 02:35, Tijl Coosemans wrote: On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: On May 24, 2014, at 5:53 PM, Warner Losh i...@bsdimp.com wrote: On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that’s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can install packages for on this machine, which is what you really want to know. So I’m having trouble connecting the dots between this and what you are saying here. I still am absolutely flabbergasted why the MACHINE_ARCH names aren’t necessary and sufficient for packaging. I’ve yet to see any coherent reason to not use them. Why do I care that they match? Good question. When I was doing FreeNAS, I looked at integrating pkgng into nanobsd. At the time this was quite difficult because every single architecture name was different between pkgng and MACHINE_ARCH. This would mean I’d have to drag around a huge table to know how to translate one to the other (there was no simple regex either, and things like mipsn32 wouldn’t have fit into the scheme at the time). I would very much like us to see us keep these names in sync and avoid large translation tables that are difficult to maintain. Now, do you need to get it from uname -p? No. If you want to parse elf files to get it, that’s fine, so long as the names map directly to the MACHINE_ARCH names that we’ve been using for years. They completely describe the universe of supported platforms. Are they perfect? No, around the edge there may be an odd-ball that’s possible to build, but is unsupported and likely doesn’t work at all. Have we learned from these mistakes? Yes. Anything that’s actively supported has a proper name. This name is needed, btw, so that any machine can self-host, a nice feature of the /usr/src system. ABI consists of the following elements: - OS - OS ABI version (major version number in FreeBSD) These two are encoded in FreeBSD and major version. There’s no problem encoding these in the package architecture string. They are easily scriptable and totally obvious to FreeBSD users and pose no problems. Nobody is opposed to these, and actually they are rather a good idea. - instruction set - programming model (ILP32 or LP64) - byte order (little/big endian) These three are encoded in MACHINE_ARCH and have been for quite some time. And you forgot several things as well: register conventions, calling conventions, stack alignment, struct alignment, pointer conversion conventions, address space layout, page size constraints, etc. There are simply far too many to try to break down like you are trying to do. And that’s even before we get into shared library conventions... I didn't forget them, I just restricted it to the elements that came up so far. All these extra elements are like byte order: you use only one of each per combination of the first four fields so they can be discarded. Things like calling conventions and register use can be considered part of the programming model. The amd64 programming models that matter to FreeBSD (both ILP32 and LP64) are documented in the System V Application Binary Interface AMD64 Architecture Processor Supplement. These are almost orthogonal dimensions in the sense that almost any combination is possible. (A combination that isn't possible is a 32-bit instruction set with LP64.) All of these items are encoded in MACHINE_ARACH and have been for at least a decade. There’s no new argument here. If they were actually orthogonal, then that would be one thing. But they aren’t. They are all closely interrelated and we only support a vanishingly small number of possible conventions. Combinatorically, it can be hundreds. Practically, it is usually only a handful. What you are asking for now is to combine two dimensions into one and combination in this case means multiplication so if you have 3 instruction sets and 2 programming models, the combined dimension needs 6 different values. You need to make the case for why you think this is a good idea. Because uanme has to be 6 different things so the right binaries are built. It is really that simple. Uname is a per system (or per jail) setting. Whether you then want a 32-bit or 64-bit address space is a separate per program
Re: svn commit: r266553 - head/release/scripts
On May 26, 2014, at 4:18 PM, Tijl Coosemans t...@freebsd.org wrote: On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: On May 26, 2014, at 8:39 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/26/14 02:35, Tijl Coosemans wrote: On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: On May 24, 2014, at 5:53 PM, Warner Losh i...@bsdimp.com wrote: On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that’s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can install packages for on this machine, which is what you really want to know. So I’m having trouble connecting the dots between this and what you are saying here. I still am absolutely flabbergasted why the MACHINE_ARCH names aren’t necessary and sufficient for packaging. I’ve yet to see any coherent reason to not use them. Why do I care that they match? Good question. When I was doing FreeNAS, I looked at integrating pkgng into nanobsd. At the time this was quite difficult because every single architecture name was different between pkgng and MACHINE_ARCH. This would mean I’d have to drag around a huge table to know how to translate one to the other (there was no simple regex either, and things like mipsn32 wouldn’t have fit into the scheme at the time). I would very much like us to see us keep these names in sync and avoid large translation tables that are difficult to maintain. Now, do you need to get it from uname -p? No. If you want to parse elf files to get it, that’s fine, so long as the names map directly to the MACHINE_ARCH names that we’ve been using for years. They completely describe the universe of supported platforms. Are they perfect? No, around the edge there may be an odd-ball that’s possible to build, but is unsupported and likely doesn’t work at all. Have we learned from these mistakes? Yes. Anything that’s actively supported has a proper name. This name is needed, btw, so that any machine can self-host, a nice feature of the /usr/src system. ABI consists of the following elements: - OS - OS ABI version (major version number in FreeBSD) These two are encoded in FreeBSD and major version. There’s no problem encoding these in the package architecture string. They are easily scriptable and totally obvious to FreeBSD users and pose no problems. Nobody is opposed to these, and actually they are rather a good idea. - instruction set - programming model (ILP32 or LP64) - byte order (little/big endian) These three are encoded in MACHINE_ARCH and have been for quite some time. And you forgot several things as well: register conventions, calling conventions, stack alignment, struct alignment, pointer conversion conventions, address space layout, page size constraints, etc. There are simply far too many to try to break down like you are trying to do. And that’s even before we get into shared library conventions... I didn't forget them, I just restricted it to the elements that came up so far. All these extra elements are like byte order: you use only one of each per combination of the first four fields so they can be discarded. Things like calling conventions and register use can be considered part of the programming model. The amd64 programming models that matter to FreeBSD (both ILP32 and LP64) are documented in the System V Application Binary Interface AMD64 Architecture Processor Supplement. Well yes and no. n32 and n64 have vastly different register conventions than o32. But we’re talking about something that’s off in the weeds. It doesn’t matter. It also doesn’t address my basic thesis “MACHINE_ARCH is enough” which you’ve not shown a coherent example of where it isn’t. These are almost orthogonal dimensions in the sense that almost any combination is possible. (A combination that isn't possible is a 32-bit instruction set with LP64.) All of these items are encoded in MACHINE_ARACH and have been for at least a decade. There’s no new argument here. If they were actually orthogonal, then that would be one thing. But they aren’t. They are all closely interrelated and we only support a vanishingly small number of possible conventions. Combinatorically, it can be hundreds. Practically, it is usually only a handful. What you are asking for now is to combine two dimensions into one and combination in this case means multiplication so if you have 3 instruction sets and 2
Re: svn commit: r266553 - head/release/scripts
On 05/26/14 16:02, dte...@freebsd.org wrote: -Original Message- From: owner-src-committ...@freebsd.org [mailto:owner-src- committ...@freebsd.org] On Behalf Of Nathan Whitehorn Sent: Monday, May 26, 2014 7:40 AM To: Tijl Coosemans; Warner Losh Cc: Ian Lepore; Baptiste Daroussin; svn-src-head@freebsd.org; Glen Barber; svn-src-...@freebsd.org; src-committ...@freebsd.org Subject: Re: svn commit: r266553 - head/release/scripts On 05/26/14 02:35, Tijl Coosemans wrote: On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: On May 24, 2014, at 5:53 PM, Warner Losh i...@bsdimp.com wrote: On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that?s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can install packages for on this machine, which is what you really want to know. So I?m having trouble connecting the dots between this and what you are saying here. I still am absolutely flabbergasted why the MACHINE_ARCH names aren?t necessary and sufficient for packaging. I?ve yet to see any coherent reason to not use them. Why do I care that they match? Good question. When I was doing FreeNAS, I looked at integrating pkgng into nanobsd. At the time this was quite difficult because every single architecture name was different between pkgng and MACHINE_ARCH. This would mean I?d have to drag around a huge table to know how to translate one to the other (there was no simple regex either, and things like mipsn32 wouldn?t have fit into the scheme at the time). I would very much like us to see us keep these names in sync and avoid large translation tables that are difficult to maintain. Now, do you need to get it from uname -p? No. If you want to parse elf files to get it, that?s fine, so long as the names map directly to the MACHINE_ARCH names that we?ve been using for years. They completely describe the universe of supported platforms. Are they perfect? No, around the edge there may be an odd-ball that?s possible to build, but is unsupported and likely doesn?t work at all. Have we learned from these mistakes? Yes. Anything that?s actively supported has a proper name. This name is needed, btw, so that any machine can self-host, a nice feature of the /usr/src system. ABI consists of the following elements: - OS - OS ABI version (major version number in FreeBSD) - instruction set - programming model (ILP32 or LP64) - byte order (little/big endian) These are almost orthogonal dimensions in the sense that almost any combination is possible. (A combination that isn't possible is a 32-bit instruction set with LP64.) What you are asking for now is to combine two dimensions into one and combination in this case means multiplication so if you have 3 instruction sets and 2 programming models, the combined dimension needs 6 different values. You need to make the case for why you think this is a good idea. For the past 20 years we got away with this because on every installation of FreeBSD we only used one programming model at a time. This is still the case for byte order of course. What I'm saying is to keep the option open for installations with multiple programming models, where most binaries could use ILP32 and only the ones that actually need a 64-bit address space use LP64. You query the instruction set using uname and the programming models using getconf. I suppose you could replace the x86 in the pkg scheme with i386/amd64, but then you'd still be talking about i386:32, amd64:32 and amd64:64 instead of x86:32, x86:x32 and x86:64. No. We support multiple models now and have for ten years. That's what MACHINE_ARCH is for: it defines the choice of the last three things you list above. Specifically, a shared value of MACHINE_ARCH guarantees and OS version guarantees, in FreeBSD-land, complete binary compatibility of executables. Kernels support multiple ones, in general (e.g. i386 binaries on amd64, powerpc binaries on powerpc64). They may support more in the future (x32 on amd64, potentially even cross-endian binaries). We have a nice flexible scheme in FreeBSD for supporting this. If you want to find out the list of the things the installed kernel can run, check the kern.supported_archs sysctl. Simple. These strings are just as expressive as the ones in pkg. They are the standard. They're what external build systems test against, what the src, doc, and ports trees use to define what to do universally. It's what users
Re: svn commit: r266553 - head/release/scripts
On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. The advantage of the pkg scheme is that it has a formal structure. That's what makes it flexible, extensible, machine parsable, etc. I'd rather see the rest of FreeBSD adopt this scheme than that pkg would have to adopt the informal names. The use of x86 instead of i386/amd64 is part of the idea to merge more of sys/i386 and sys/amd64 into sys/x86 and eventually define MACHINE as x86. Patterns like freebsd:9:* will probably become more prevalent when support for subpackages is added. Some of the subpackages (like documentation) will be ABI independent. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. The advantage of the pkg scheme is that it has a formal structure. That's what makes it flexible, extensible, machine parsable, etc. I'd rather see the rest of FreeBSD adopt this scheme than that pkg would have to adopt the informal names. The use of x86 instead of i386/amd64 is part of the idea to merge more of sys/i386 and sys/amd64 into sys/x86 and eventually define MACHINE as x86. We need one set of platform names. FreeBSD has used MACHINE_ARCH values for a very long time for this and uses them absolutely everywhere in the ports and src trees -- except for the internals of pkg. I don't think they can be changed at this point, even if some names are slightly cryptic. They are at least internally consistent, and that's a huge win. I agree it will be good to get i386 and amd64 sharing a MACHINE_CPUARCH and MACHINE value at some point, but it's not really related to the issue of uniform ABI tags. Patterns like freebsd:9:* will probably become more prevalent when support for subpackages is added. Some of the subpackages (like documentation) will be ABI independent. That wouldn't be affected by using standard ABI identifiers. On a more general note, any flexibility provided by things like x86:64 is illusory anyway. Not all amd64 systems can run i386 binaries. Some ia64 systems *can* run i386 binaries. i386 systems cannot run amd64 binaries on FreeBSD -- but on OS X they sometimes can. Some powerpc systems can run little-endian binaries, but not all (G5s notably cannot). It's just not possible to make static decisions here and there doesn't seem to be any reason to invent a fragile alternative definition of ABI identifiers that is at variance with everything else in FreeBSD for apparently the sole purpose of facilitating something that is intrinsically fragile and can't be relied upon. This is especially true given that the kernel *does* tell you, in MACHINE_ARCH terms, what binaries any given system can run -- but for pkg to use that information, it currently needs a mapping table! -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On Sat, 2014-05-24 at 18:53 +0200, Tijl Coosemans wrote: On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. All you have to do is look at the plethora of ARM ABIs we support (and the corresponding separate kernel for each) to see the falseness of that last sentence. ARM variations include v4 vs v6, OABI vs EABI (calling and register usage standards), hard vs soft float, little vs big endian. Virtually all combinations of those are possible (there are a few combos we don't support), and each one has its own MACHINE_ARCH. -- Ian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 24, 2014, at 8:59 AM, Tijl Coosemans t...@freebsd.org wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. ABIs like x32 would not have a MACHINE_ARCH of “amd64” but would have a MACHINE_ARCH of “x32”. This is exactly what we do with mips today. So this ins’t an argument for not using MACHINE_ARCH directly, rather than having an arbitrary mapping (which is the problem with the proposed scheme). MACHINE_ARCH, as it stands in FreeBSD, uniquely defines the ABI (modulo occasional bugs that are fixed). The advantage of the pkg scheme is that it has a formal structure. That's what makes it flexible, extensible, machine parsable, etc. I'd rather see the rest of FreeBSD adopt this scheme than that pkg would have to adopt the informal names. The use of x86 instead of i386/amd64 is part of the idea to merge more of sys/i386 and sys/amd64 into sys/x86 and eventually define MACHINE as x86”. MACHINE and MACHINE_ARCH are different. Please don’t confuse them. Patterns like freebsd:9:* will probably become more prevalent when support for subpackages is added. Some of the subpackages (like documentation) will be ABI independent. True, but not relevant to the machine name you use. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r266553 - head/release/scripts
On 05/24/14 11:23, Ian Lepore wrote: On Sat, 2014-05-24 at 18:53 +0200, Tijl Coosemans wrote: On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. All you have to do is look at the plethora of ARM ABIs we support (and the corresponding separate kernel for each) to see the falseness of that last sentence. ARM variations include v4 vs v6, OABI vs EABI (calling and register usage standards), hard vs soft float, little vs big endian. Virtually all combinations of those are possible (there are a few combos we don't support), and each one has its own MACHINE_ARCH. Exactly. This doesn't rely on the kernel either. The hw.machine_arch sysctl (what uname -p returns) gives the ABI of the calling binary rather than the kernel. So if you use a 32-bit uname (e.g. in a chroot) on an amd64 host, you get i386. The same will be true if and when we support a 32-bit amd64 userland -- even if there is no x32 kernel, an x32 uname will return x32 (or amd32 or whatever it ends up being called). That string will also appear in kern.supported_archs. MACHINE_ARCH completely defines the ABI, without exception, because, as a matter of policy, that's what we have defined it to mean. If there are any circumstances where it does not -- and none have been offered so far -- those are simply bugs that need fixing. -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 24, 2014, at 10:53 AM, Tijl Coosemans t...@freebsd.org wrote: On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. I think we’re talking past each other here. It absolutely would be MACHINE_ARCH == x32. That would be the name. And you could have an x32 kernel, if you wanted. When we’re building in the FreeBSD build system, we’d absolutely need a second MACHINE_ARCH. What you are doing here is confuse MACHINE_ARCH with MACHINE_CPUARCH. If you look at how we handle this for MIPS you’ll see we do exactly this for MIPS N32, which is exact likely x32. You don’t have to have a n32 kernel to run n32 binaries (but in MIPS’ case you can, if you want). Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r266553 - head/release/scripts
On May 24, 2014, at 12:57 PM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/24/14 11:23, Ian Lepore wrote: On Sat, 2014-05-24 at 18:53 +0200, Tijl Coosemans wrote: On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. All you have to do is look at the plethora of ARM ABIs we support (and the corresponding separate kernel for each) to see the falseness of that last sentence. ARM variations include v4 vs v6, OABI vs EABI (calling and register usage standards), hard vs soft float, little vs big endian. Virtually all combinations of those are possible (there are a few combos we don't support), and each one has its own MACHINE_ARCH. Exactly. This doesn't rely on the kernel either. The hw.machine_arch sysctl (what uname -p returns) gives the ABI of the calling binary rather than the kernel. So if you use a 32-bit uname (e.g. in a chroot) on an amd64 host, you get i386. The same will be true if and when we support a 32-bit amd64 userland -- even if there is no x32 kernel, an x32 uname will return x32 (or amd32 or whatever it ends up being called). That string will also appear in kern.supported_archs. Yes, the supported_arch sysctl is 100% based on MACHINE_ARCH names. That’s why we added it: so that the packaging system could use it to offer which packages to install. MACHINE_ARCH completely defines the ABI, without exception, because, as a matter of policy, that's what we have defined it to mean. If there are any circumstances where it does not -- and none have been offered so far -- those are simply bugs that need fixing. Also, the list of ABIs we support is exported using MACHINE_ARCH names, Those are the standard names that we use to describe everything else ABI in the system. If there is a pkg layer translation, then it absolutely must support translation to/from these names in the pkg layer so that all the scripts that cope
Re: svn commit: r266553 - head/release/scripts
On Sat, 24 May 2014 11:57:44 -0700 Nathan Whitehorn wrote: On 05/24/14 11:23, Ian Lepore wrote: On Sat, 2014-05-24 at 18:53 +0200, Tijl Coosemans wrote: On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. All you have to do is look at the plethora of ARM ABIs we support (and the corresponding separate kernel for each) to see the falseness of that last sentence. ARM variations include v4 vs v6, OABI vs EABI (calling and register usage standards), hard vs soft float, little vs big endian. Virtually all combinations of those are possible (there are a few combos we don't support), and each one has its own MACHINE_ARCH. Exactly. This doesn't rely on the kernel either. The hw.machine_arch sysctl (what uname -p returns) gives the ABI of the calling binary rather than the kernel. So if you use a 32-bit uname (e.g. in a chroot) on an amd64 host, you get i386. The same will be true if and when we support a 32-bit amd64 userland -- even if there is no x32 kernel, an x32 uname will return x32 (or amd32 or whatever it ends up being called). That string will also appear in kern.supported_archs. There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: On Sat, 24 May 2014 11:57:44 -0700 Nathan Whitehorn wrote: On 05/24/14 11:23, Ian Lepore wrote: On Sat, 2014-05-24 at 18:53 +0200, Tijl Coosemans wrote: On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. All you have to do is look at the plethora of ARM ABIs we support (and the corresponding separate kernel for each) to see the falseness of that last sentence. ARM variations include v4 vs v6, OABI vs EABI (calling and register usage standards), hard vs soft float, little vs big endian. Virtually all combinations of those are possible (there are a few combos we don't support), and each one has its own MACHINE_ARCH. Exactly. This doesn't rely on the kernel either. The hw.machine_arch sysctl (what uname -p returns) gives the ABI of the calling binary rather than the kernel. So if you use a 32-bit uname (e.g. in a chroot) on an amd64 host, you get i386. The same will be true if and when we support a 32-bit amd64 userland -- even if there is no x32 kernel, an x32 uname will return x32 (or amd32 or whatever it ends up being called). That string will also appear in kern.supported_archs. There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that’s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can install packages for on this machine, which is what you really want to know.
Re: svn commit: r266553 - head/release/scripts
On May 24, 2014, at 5:53 PM, Warner Losh i...@bsdimp.com wrote: On May 24, 2014, at 5:13 PM, Tijl Coosemans t...@freebsd.org wrote: On Sat, 24 May 2014 11:57:44 -0700 Nathan Whitehorn wrote: On 05/24/14 11:23, Ian Lepore wrote: On Sat, 2014-05-24 at 18:53 +0200, Tijl Coosemans wrote: On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: On 05/24/14 07:59, Tijl Coosemans wrote: On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? ABI isn't just about the instruction set. It's also about the sizes of C types (like pointers). If I remember correctly, the pkg scheme was chosen to allow for ABIs like x32 which use the 64 bit instruction set with 32 bit pointers. MACHINE_ARCH would also be amd64 in this case. No, it wouldn't. MACHINE_ARCH would be something else (x32, probably) in such cases. MACHINE_ARCH (and uname -p, which reports it) is the FreeBSD ABI identifier and encodes 100% of the ABI information. This would be true even if there is never an x32 kernel. No, there's no such thing as an x32 kernel. It's an amd64 kernel that supports a second userland ABI. In C preprocessor terms they are distinguished by (__amd64__ _LP64) and (__amd64__ !_LP64). uname -p gives you the processor architecture (the __amd64__ bit) but then you can still choose the sizes of standard C types (the _LP64 bit). So far we've always had one ABI per processor architecture but this is not strictly necessary. All you have to do is look at the plethora of ARM ABIs we support (and the corresponding separate kernel for each) to see the falseness of that last sentence. ARM variations include v4 vs v6, OABI vs EABI (calling and register usage standards), hard vs soft float, little vs big endian. Virtually all combinations of those are possible (there are a few combos we don't support), and each one has its own MACHINE_ARCH. Exactly. This doesn't rely on the kernel either. The hw.machine_arch sysctl (what uname -p returns) gives the ABI of the calling binary rather than the kernel. So if you use a 32-bit uname (e.g. in a chroot) on an amd64 host, you get i386. The same will be true if and when we support a 32-bit amd64 userland -- even if there is no x32 kernel, an x32 uname will return x32 (or amd32 or whatever it ends up being called). That string will also appear in kern.supported_archs. There isn't necessarily any chroot environment. There's one kernel, two equally valid ABIs (ILP32 and LP64) and any binary like uname might use either of them. If uname -p returns a different result depending on which of these two ABIs it was compiled for that could be a problem for any script that uses it. Well, it depends on what you want to do with the script, eh? If you want to know the ABI of the native binary uname, that’s one thing. But if you want to know the supported ABIs, you are doing it wrong by using uname. You should be using sysctl kern.supported_abi. That will tell you all the ABIs that you can
Re: svn commit: r266553 - head/release/scripts
Is there any chance of finally switching the pkg abi identifiers to just
be uname -p?
-Nathan
On 05/22/14 12:22, Glen Barber wrote:
Author: gjb
Date: Thu May 22 19:22:03 2014
New Revision: 266553
URL: http://svnweb.freebsd.org/changeset/base/266553
Log:
Add forward-compatibility glue with pkg-1.3:
- Use ASSUME_ALWAYS_YES=YES instead of ASSUME_ALWAYS_YES=1
since pkg-1.3 expects yes or true values.
- Before exporting PKG_ABI, strip extra characters from what
is parsed from 'pkg -vv'. This causes problems further down
when creating the packages directory for inclusion on the
dvd1.iso. Previously PKG_ABI would be 'freebsd:9:x86:64',
but now is 'freebsd:9:x86:64;' in pkg-1.3
Tested on: stable/9@r265858 with ports-mgmt/pkg-devel
MFC After: 3 days
Sponsored by:The FreeBSD Foundation
Modified:
head/release/scripts/pkg-stage.sh
Modified: head/release/scripts/pkg-stage.sh
==
--- head/release/scripts/pkg-stage.sh Thu May 22 18:22:02 2014
(r266552)
+++ head/release/scripts/pkg-stage.sh Thu May 22 19:22:03 2014
(r266553)
@@ -5,7 +5,7 @@
set -e
-export ASSUME_ALWAYS_YES=1
+export ASSUME_ALWAYS_YES=YES
export PKG_DBDIR=/tmp/pkg
export PERMISSIVE=YES
export REPO_AUTOUPDATE=NO
@@ -40,7 +40,10 @@ if [ ! -x /usr/local/sbin/pkg ]; then
/usr/bin/make -C /usr/ports/ports-mgmt/pkg install clean
fi
-export PKG_ABI=$(pkg -vv | grep ^ABI | awk '{print $3}')
+PKG_ABI=$(pkg -vv | grep ^ABI | awk '{print $3}')
+PKG_ABI=${PKG_ABI%\;}
+PKG_ABI=${PKG_ABI#\}
+export PKG_ABI
export PKG_CACHEDIR=dvd/packages/${PKG_ABI}
/bin/mkdir -p ${PKG_CACHEDIR}
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
It would certainly make my life easier. But until such thing is done in
pkg(7) code, I have to put things in a directory structure that pkg(7)
can just use for on-disc packages.
Glen
On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote:
Is there any chance of finally switching the pkg abi identifiers to just be
uname -p?
-Nathan
On 05/22/14 12:22, Glen Barber wrote:
Author: gjb
Date: Thu May 22 19:22:03 2014
New Revision: 266553
URL: http://svnweb.freebsd.org/changeset/base/266553
Log:
Add forward-compatibility glue with pkg-1.3:
- Use ASSUME_ALWAYS_YES=YES instead of ASSUME_ALWAYS_YES=1
since pkg-1.3 expects yes or true values.
- Before exporting PKG_ABI, strip extra characters from what
is parsed from 'pkg -vv'. This causes problems further down
when creating the packages directory for inclusion on the
dvd1.iso. Previously PKG_ABI would be 'freebsd:9:x86:64',
but now is 'freebsd:9:x86:64;' in pkg-1.3
Tested on:stable/9@r265858 with ports-mgmt/pkg-devel
MFC After:3 days
Sponsored by: The FreeBSD Foundation
Modified:
head/release/scripts/pkg-stage.sh
Modified: head/release/scripts/pkg-stage.sh
==
--- head/release/scripts/pkg-stage.shThu May 22 18:22:02 2014
(r266552)
+++ head/release/scripts/pkg-stage.shThu May 22 19:22:03 2014
(r266553)
@@ -5,7 +5,7 @@
set -e
-export ASSUME_ALWAYS_YES=1
+export ASSUME_ALWAYS_YES=YES
export PKG_DBDIR=/tmp/pkg
export PERMISSIVE=YES
export REPO_AUTOUPDATE=NO
@@ -40,7 +40,10 @@ if [ ! -x /usr/local/sbin/pkg ]; then
/usr/bin/make -C /usr/ports/ports-mgmt/pkg install clean
fi
-export PKG_ABI=$(pkg -vv | grep ^ABI | awk '{print $3}')
+PKG_ABI=$(pkg -vv | grep ^ABI | awk '{print $3}')
+PKG_ABI=${PKG_ABI%\;}
+PKG_ABI=${PKG_ABI#\}
+export PKG_ABI
export PKG_CACHEDIR=dvd/packages/${PKG_ABI}
/bin/mkdir -p ${PKG_CACHEDIR}
pgpFPuEvrnbJv.pgp
Description: PGP signature
Re: svn commit: r266553 - head/release/scripts
On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt pgpGgLO0EGlis.pgp Description: PGP signature
Re: svn commit: r266553 - head/release/scripts
On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* regards, Bapt pgpMd7UlJ4pac.pgp Description: PGP signature
Re: svn commit: r266553 - head/release/scripts
On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. and last thing is the current build packages should just work meaning that we would need to have a kind of mapping table Sure, as a compat measure. No reason to lock it in forever. You could also detect old-style strings with a warning and install them unconditionally. It's not a big deal. -nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On 2014-05-23 12:11, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. Symlinks are irrelevant for pkg. It may fetch based on ABI in the URL, but once it opens the package it also compares the ABI from the package and repository to the ABI of /bin/sh on the system. So the symlink wouldn't help. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. and last thing is the current build packages should just work meaning that we would need to have a kind of mapping table Sure, as a compat measure. No reason to lock it in forever. You could also detect old-style strings with a warning and install them unconditionally. It's not a big deal. -nathan -- Regards, Bryan Drewery ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. and last thing is the current build packages should just work meaning that we would need to have a kind of mapping table Sure, as a compat measure. No reason to lock it in forever. You could also detect old-style strings with a warning and install them unconditionally. It's not a big deal. sure but one has to write it :) -nathan regards, Bapt pgpRw5OLQiGd9.pgp Description: PGP signature
Re: svn commit: r266553 - head/release/scripts
On 05/23/14 10:24, Bryan Drewery wrote: On 2014-05-23 12:11, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. Symlinks are irrelevant for pkg. It may fetch based on ABI in the URL, but once it opens the package it also compares the ABI from the package and repository to the ABI of /bin/sh on the system. So the symlink wouldn't help. That is a highly questionable design choice. Why not just check MACHINE_ARCH? In any case, packages only exist for i386 and amd64 in the wild in a supported way. Those two can be special cased for compatibility in about two lines of code if you're worried about this. -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. and last thing is the current build packages should just work meaning that we would need to have a kind of mapping table Sure, as a compat measure. No reason to lock it in forever. You could also detect old-style strings with a warning and install them unconditionally. It's not a big deal. sure but one has to write it :) That's fine. I'm happy to. -Nathan ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work and last thing is the current build packages should just work meaning that we would need to have a kind of mapping table Sure, as a compat measure. No reason to lock it in forever. You could also detect old-style
Re: svn commit: r266553 - head/release/scripts
On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what kernel will be running in there and how it will be configured? You don't. IA64 can -- sometimes -- run i386 binaries, for example. amd64 may or may not be able to run i386, depending on kernel options. In any
Re: svn commit: r266553 - head/release/scripts
On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what kernel will be running in there and how it will be configured? You don't. IA64 can -- sometimes -- run i386 binaries, for example.
Re: svn commit: r266553 - head/release/scripts
Bryan Drewery wrote this message on Fri, May 23, 2014 at 16:34 -0500: On 2014-05-23 16:19, Nathan Whitehorn wrote: Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what kernel will be running in there and how it will be configured? You don't. IA64 can -- sometimes -- run i386 binaries, for example. amd64 may or may not be able to run i386, depending on kernel options. You're assuming that you would only use a chroot to RUN things. This is also useful for building images. Install a world into a chroot, run pkg -c install whatever and it picks the right ABI. Just an example. Will there be a mode to output an mtree file (or append to one) w/ the correct permissions? We are slowly moving to not requiring root to build images, and if you now need to have root to chroot or chown files, then it won't be as useful for building images... Right now I'm looking at tools/tools/makeroot that allows merging of mtree files, which will allow us to easily customize the built image.. -- John-Mark Gurney Voice: +1 415 225 5579 All that I will do, has been done, All that I have, has not. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On 05/23/14 14:34, Bryan Drewery wrote: On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what kernel will be running in there and how it will be configured? You don't. IA64 can --
Re: svn commit: r266553 - head/release/scripts
On May 23, 2014, at 3:34 PM, Bryan Drewery bdrew...@freebsd.org wrote: On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what kernel will be running in there
Re: svn commit: r266553 - head/release/scripts
On Fri, May 23, 2014 at 04:15:05PM -0700, Nathan Whitehorn wrote: On 05/23/14 14:34, Bryan Drewery wrote: On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very
Re: svn commit: r266553 - head/release/scripts
On May 23, 2014, at 5:22 PM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 04:15:05PM -0700, Nathan Whitehorn wrote: On 05/23/14 14:34, Bryan Drewery wrote: On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work
Re: svn commit: r266553 - head/release/scripts
On May 23, 2014, at 10:20 AM, Baptiste Daroussin b...@freebsd.org wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi - The different float abi (even if only one is supported for now others are being worked on) - little endian vs big endian All of those are encoded in the MACHINE_ARCH + freebsd version, no exceptions on supported architectures that are tier 2 or higher. This seems like a weak reason. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* Will there be a program to convert this new, special invention to the standard that we’ve used for the past 20 years? If you need the flexibility, which I’m not entirely sure I’ve seen a good use case for. When would you have a x86 binary package? Wouldn’t it be either i386 or amd64? Warner ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On May 23, 2014, at 9:52 AM, Nathan Whitehorn nwhiteh...@freebsd.org wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. What is not present? I’ve explained this many times. All examples that I’ve seen offered are the result of bugs, or are for unsupported configurations. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. I’m putting my hand up. Warner ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r266553 - head/release/scripts
On 2014-05-23 17:34, Bryan Drewery wrote: On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what kernel will be running in there and how it will be configured? You don't. IA64 can --
Re: svn commit: r266553 - head/release/scripts
On 2014-05-23 18:15, Nathan Whitehorn wrote: On 05/23/14 14:34, Bryan Drewery wrote: On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what kernel will be running in there and how it
Re: svn commit: r266553 - head/release/scripts
On 05/23/14 19:33, Bryan Drewery wrote: On 2014-05-23 18:15, Nathan Whitehorn wrote: On 05/23/14 14:34, Bryan Drewery wrote: On 2014-05-23 16:19, Nathan Whitehorn wrote: On 05/23/14 12:27, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 12:01:08PM -0700, Nathan Whitehorn wrote: On 05/23/14 10:26, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 10:11:47AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:45, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 09:38:14AM -0700, Nathan Whitehorn wrote: On 05/23/14 09:20, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn wrote: On 05/23/14 08:36, Baptiste Daroussin wrote: On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn wrote: Is there any chance of finally switching the pkg abi identifiers to just be uname -p? -Nathan Keeping asking won't make it happen, I have explained a large number of time why it happened, why it is not easy for compatibility and why uname -p is still not representing the ABI we do support, and what flexibility we need that the current string offers to us. if one is willing to do the work, please be my guess, just dig into the archives and join the pkg development otherwise: no it won't happen before a while because we have way too much work on the todo and this item is stored at the very end of this todo. regards, Bapt I'm happy to do the work, and have volunteered now many times. If uname -p does not describe the ABI fully, then uname -p needs changes on the relevant platforms. Which are they? What extra flexibility does the string give you if uname -p describes the ABI completely? -Nathan just simple examples in armv6: - eabi vs oabi OABI is almost entirely dead, and will be entirely dead soon. Maybe but still for now it is there and pkg has to work now We don't provide packages for ARM. Also, no platforms have defaulted to OABI for a very long time. Not making a distinction was a deliberate decision of the ARM group, since it was meant to be a clean switchover. - The different float abi (even if only one is supported for now others are being worked on) armv6 and armv6hf - little endian vs big endian armv6 and armv6eb (though I think armv6eb support in general has been removed from the tree, but armeb is still there) what about combinaison? armv6 + eb + hf? That would be armv6hfeb, I assume, if FreeBSD actually supported big-endian ARMv6 at all, which it doesn't. These all already exist. the extras flexibilit is being able to say this binary do support freebsd i386 and amd64 in one key, freebsd:9:x86:*, or or all arches freebsd:10:* arm was en example what about mips? The same. There is mips64el, mipsel, mips, mips64, etc. that go through all possible combinations. This is true for all platforms and has been for ages. There was a brief period (2007-2010, I think) where some Tier-3 embedded platforms didn't have enough options, but that era was obscure and is long past. The second one already would work, wouldn't it? Just replacing x86:64 with amd64 won't change anything. The first has to be outweighed by being able to reliably figure out where to fetch from without a lookup table. We also added the kern.supported_archs sysctl last year to all branches to enable figuring out which architectures a given running kernel supports (e.g. amd64 and i386 on most amd64 systems). This was designed specifically to help pkg figure out what packages it can install. I know, it means that we can switch only when freebsd 8 and 9 are EOL which means in a couple of years Why does it mean that? That doesn't make sense. A couple of symlinks on the FTP server ensure compatibility. For the sysctl, it has been merged all the back to 7. So We can switch after 8.4 death which is a good news (except if you say that it is in 8.4) It means we can do it now. Very few people install i386 packages on amd64 anyway. It means people with very old releases on old branches might face a warning in an unusual situation. Not a big deal. Since we only provide i386 and amd64 packages anyway, this is also a trivial special case if you really want that. And it defeats cross installation (which is the reason why the ABI supported is read from a binary and not from kernel) No. That's the point of the sysctl. I'm speaking of installing packages in a arm chroot on a amd64 host I will need to know what arch could be supported by the content of the chroot. uname -p in the chroot (I guess this is with qemu) should return the right answer, just as it does with an i386 chroot. If it doesn't, something is broken in the qemu user mode support. nope that is not with qemu it is basically cross buildworld, install in a destdir, install packages in that destdir which is a very common usage that a lot do expect to work Knowing a priori which architectures are supported by a chroot based on ELF type of /bin/sh doesn't even work. How do you know what
svn commit: r266553 - head/release/scripts
Author: gjb
Date: Thu May 22 19:22:03 2014
New Revision: 266553
URL: http://svnweb.freebsd.org/changeset/base/266553
Log:
Add forward-compatibility glue with pkg-1.3:
- Use ASSUME_ALWAYS_YES=YES instead of ASSUME_ALWAYS_YES=1
since pkg-1.3 expects yes or true values.
- Before exporting PKG_ABI, strip extra characters from what
is parsed from 'pkg -vv'. This causes problems further down
when creating the packages directory for inclusion on the
dvd1.iso. Previously PKG_ABI would be 'freebsd:9:x86:64',
but now is 'freebsd:9:x86:64;' in pkg-1.3
Tested on:stable/9@r265858 with ports-mgmt/pkg-devel
MFC After:3 days
Sponsored by: The FreeBSD Foundation
Modified:
head/release/scripts/pkg-stage.sh
Modified: head/release/scripts/pkg-stage.sh
==
--- head/release/scripts/pkg-stage.sh Thu May 22 18:22:02 2014
(r266552)
+++ head/release/scripts/pkg-stage.sh Thu May 22 19:22:03 2014
(r266553)
@@ -5,7 +5,7 @@
set -e
-export ASSUME_ALWAYS_YES=1
+export ASSUME_ALWAYS_YES=YES
export PKG_DBDIR=/tmp/pkg
export PERMISSIVE=YES
export REPO_AUTOUPDATE=NO
@@ -40,7 +40,10 @@ if [ ! -x /usr/local/sbin/pkg ]; then
/usr/bin/make -C /usr/ports/ports-mgmt/pkg install clean
fi
-export PKG_ABI=$(pkg -vv | grep ^ABI | awk '{print $3}')
+PKG_ABI=$(pkg -vv | grep ^ABI | awk '{print $3}')
+PKG_ABI=${PKG_ABI%\;}
+PKG_ABI=${PKG_ABI#\}
+export PKG_ABI
export PKG_CACHEDIR=dvd/packages/${PKG_ABI}
/bin/mkdir -p ${PKG_CACHEDIR}
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
