Re: Upcoming ABI Breakage in RELENG_7
On Tue, Jul 29, 2008 at 10:45 AM, Ken Smith [EMAIL PROTECTED] wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be implemented without ABI breakage, and it is decided that the fix warrants the impact of the ABI breakage. We have one of those situations coming along for RELENG_7 (what will become FreeBSD 7.1). The ABI breakage should only impact kernel modules that are not part of the baseline system (those will be patched by the MFC) which deal with advisory locks. As such the impact should not cause many people problems. The work that will be MFCed fixes issues with filesystem advisory locks, and moves the advisory locks list from filesystem-private data structures into the vnode structure. The MFC will be done by Kostantin Belousov some time this coming Friday (August 1st, 2008) if you have concerns and want to watch for it. Just updated to 7-STABLE last night and found that Truecrypt (which uses a combination of fuse and mdconfig to mount its encrypted volumes) is now completely unable to mount its encrypted volumes. I've rebuilt fusefs-kmod and fusefs-libs as well as Truecrypt to no avail. As Truecrypt is implementing an encrypted file system, I am assuming at this point that the kernel ABI breakage may be what is causing this new problem. Are the any calls in particular that I should be looking for in the Truecrypt code to see where it might be choking on the new kernel ABI? I've briefly reviewed some of the changes in r181119 but lack the experience to discern anything useful from them. I'd like to file an issue with the Truecrypt devs if I can get an idea of where the breakage might be. Thanks, Matt Thanks. -- Ken Smith - From there to here, from here to | [EMAIL PROTECTED] there, funny things are everywhere. | - Theodore Geisel | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
Matt wrote: Just updated to 7-STABLE last night and found that Truecrypt (which uses a combination of fuse and mdconfig to mount its encrypted volumes) is now completely unable to mount its encrypted volumes. I've rebuilt fusefs-kmod and fusefs-libs as well as Truecrypt to no avail. Silly question, but did you build AND install both a new world AND a new kernel before you rebuilt your ports? Doug -- This .signature sanitized for your protection ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Tue, Aug 5, 2008 at 5:09 PM, Doug Barton [EMAIL PROTECTED] wrote: Matt wrote: Just updated to 7-STABLE last night and found that Truecrypt (which uses a combination of fuse and mdconfig to mount its encrypted volumes) is now completely unable to mount its encrypted volumes. I've rebuilt fusefs-kmod and fusefs-libs as well as Truecrypt to no avail. Silly question, but did you build AND install both a new world AND a new kernel before you rebuilt your ports? Yes - full, clean (deleted /usr/obj/* before build) build on world + kernel. I've run it through a couple ktrace runs looking for obvious problems, but haven't found anything that I can interpret yet. It appears the failures are triggering around some file-descriptor errors, but nothing that makes any sense yet. I'll look at things a little closer when I get some more time - need to boot into a backup copy of the 7-RELEASE system to get some baselines. Matt Doug -- This .signature sanitized for your protection ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
* Ken Smith [EMAIL PROTECTED] [080729 08:47] wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be implemented without ABI breakage, and it is decided that the fix warrants the impact of the ABI breakage. We have one of those situations coming along for RELENG_7 (what will become FreeBSD 7.1). The ABI breakage should only impact kernel modules that are not part of the baseline system (those will be patched by the MFC) which deal with advisory locks. As such the impact should not cause many people problems. The work that will be MFCed fixes issues with filesystem advisory locks, and moves the advisory locks list from filesystem-private data structures into the vnode structure. The MFC will be done by Kostantin Belousov some time this coming Friday (August 1st, 2008) if you have concerns and want to watch for it. Ken, Can you point at a cvs/svn log or two that details the change and why? Everyone else: For those confused about what ABI breakage means: It means that you'll need to recompile your kernel modules and potentially your system utilities that access kernel data structures to display statistics. It seems like in this particular case you won't need to recompile, but it's a good idea just to be safe to recompile kernel, world and any third party kernel modules you have. thank you, -Alfred ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Fri, Aug 01, 2008 at 05:44:17AM -0700, Alfred Perlstein wrote: * Ken Smith [EMAIL PROTECTED] [080729 08:47] wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be implemented without ABI breakage, and it is decided that the fix warrants the impact of the ABI breakage. We have one of those situations coming along for RELENG_7 (what will become FreeBSD 7.1). The ABI breakage should only impact kernel modules that are not part of the baseline system (those will be patched by the MFC) which deal with advisory locks. As such the impact should not cause many people problems. The work that will be MFCed fixes issues with filesystem advisory locks, and moves the advisory locks list from filesystem-private data structures into the vnode structure. The MFC will be done by Kostantin Belousov some time this coming Friday (August 1st, 2008) if you have concerns and want to watch for it. Ken, Can you point at a cvs/svn log or two that details the change and why? MFCed as r181119. See the log for all details. pgpvh5xa2WgAK.pgp Description: PGP signature
Re: Upcoming ABI Breakage in RELENG_7
On Tuesday 29 July 2008 08:45:45 Ken Smith wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be implemented without ABI breakage, and it is decided that the fix warrants the impact of the ABI breakage. We have one of those situations coming along for RELENG_7 (what will become FreeBSD 7.1). The ABI breakage should only impact kernel modules that are not part of the baseline system (those will be patched by the MFC) which deal with advisory locks. As such the impact should not cause many people problems. The work that will be MFCed fixes issues with filesystem advisory locks, and moves the advisory locks list from filesystem-private data structures into the vnode structure. The MFC will be done by Kostantin Belousov some time this coming Friday (August 1st, 2008) if you have concerns and want to watch for it. Thanks. Sometimes information gets posted to this list on the assumption that everyone understand what the writer means. This is one of those occasions!! For those of us who are not as well informed and experienced as others could someone please explain what is meant by an ABI breakage, its implications and how to deal with them. Thanks David ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Wed, 30 Jul 2008 11:47:34 +0200, David Southwell [EMAIL PROTECTED] wrote: On Tuesday 29 July 2008 08:45:45 Ken Smith wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be implemented without ABI breakage, and it is decided that the fix warrants the impact of the ABI breakage. We have one of those situations coming along for RELENG_7 (what will become FreeBSD 7.1). The ABI breakage should only impact kernel modules that are not part of the baseline system (those will be patched by the MFC) which deal with advisory locks. As such the impact should not cause many people problems. The work that will be MFCed fixes issues with filesystem advisory locks, and moves the advisory locks list from filesystem-private data structures into the vnode structure. The MFC will be done by Kostantin Belousov some time this coming Friday (August 1st, 2008) if you have concerns and want to watch for it. Thanks. Sometimes information gets posted to this list on the assumption that everyone understand what the writer means. This is one of those occasions!! For those of us who are not as well informed and experienced as others could someone please explain what is meant by an ABI breakage, its implications and how to deal with them. Thanks David Googling for ABI gives me this: http://en.wikipedia.org/wiki/Application_binary_interface That is part of what you want to know. Ronald. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Wednesday 30 July 2008 11:47:34 David Southwell wrote: On Tuesday 29 July 2008 08:45:45 Ken Smith wrote: ... For those of us who are not as well informed and experienced as others could someone please explain what is meant by an ABI breakage, its implications and how to deal with them. An Application Binary Interface (ABI) breakage means a change of a complex datatype, or a function prototype - in C usually the change of a struct in size or field sort order. This change will effect all consumers of this Interface unless they are recompiled with the changed header files to also use the new interface. The impact depends greatly on the interface that is being changed. As Ken described in the initial mail this change is in the filesystem/vnode layer interface and thus will (only) concern consumers of that ABI. The changed interface is also a kernel-only interface - that means that only 3rd party kernel modules will be affected. I personally don't know of any 3rd party modules that muck about with filesystems (for which you can't get the source). That said, you might have to rebuild stuff like fuse after the breakage has happened. I assume that port maintainers of affected (kernel module) ports will bump the port revision after the change to give you/portupgrade a hint. -- /\ Best regards, | [EMAIL PROTECTED] \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | [EMAIL PROTECTED] / \ ASCII Ribbon Campaign | Against HTML Mail and News ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Wed, Jul 30, 2008 at 02:47:34AM -0700, David Southwell wrote: On Tuesday 29 July 2008 08:45:45 Ken Smith wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be implemented without ABI breakage, and it is decided that the fix warrants the impact of the ABI breakage. We have one of those situations coming along for RELENG_7 (what will become FreeBSD 7.1). The ABI breakage should only impact kernel modules that are not part of the baseline system (those will be patched by the MFC) which deal with advisory locks. As such the impact should not cause many people problems. The work that will be MFCed fixes issues with filesystem advisory locks, and moves the advisory locks list from filesystem-private data structures into the vnode structure. The MFC will be done by Kostantin Belousov some time this coming Friday (August 1st, 2008) if you have concerns and want to watch for it. Thanks. Sometimes information gets posted to this list on the assumption that everyone understand what the writer means. This is one of those occasions!! For those of us who are not as well informed and experienced as others could someone please explain what is meant by an ABI breakage, its implications and how to deal with them. The small glitch in the announcement is use of the ABI == Application Binary Interface term, that is better be replaced by KBI == Kernel Binary Interface. No usermode breakage is expected to result from MFC. The only consequence is the need to adopt some out-of-tree filesystems, not that I am aware of any ATM. If you are the author or maintainer of such module, then you need to watch this out. pgpfRpUH7cjj4.pgp Description: PGP signature
Re: Upcoming ABI Breakage in RELENG_7
On 7/30/08, David Southwell [EMAIL PROTECTED] wrote: On Tuesday 29 July 2008 08:45:45 Ken Smith wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be implemented without ABI breakage, and it is decided that the fix warrants the impact of the ABI breakage. We have one of those situations coming along for RELENG_7 (what will become FreeBSD 7.1). The ABI breakage should only impact kernel modules that are not part of the baseline system (those will be patched by the MFC) which deal with advisory locks. As such the impact should not cause many people problems. The work that will be MFCed fixes issues with filesystem advisory locks, and moves the advisory locks list from filesystem-private data structures into the vnode structure. The MFC will be done by Kostantin Belousov some time this coming Friday (August 1st, 2008) if you have concerns and want to watch for it. Thanks. Sometimes information gets posted to this list on the assumption that everyone understand what the writer means. This is one of those occasions!! For those of us who are not as well informed and experienced as others could someone please explain what is meant by an ABI breakage, its implications and how to deal with them. ABI breakage occurs when internal data structures change (for instance, when members of the structure are removed or added). Kernel modules which expect those structures to look in a certain way will need to be recompiled. Also, depending on what data structures suffer the changes, ioctl() operations may fail, requiring a rebuild of the userland programs which issue the ioctl()s. And I'm sure that there are many other examples that I can't think of right now :) -- ~/.signature: no such file or directory ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
Am Mittwoch, 30. Juli 2008 11:47:34 schrieb David Southwell:
For those of us who are not as well informed and experienced as others
could someone please explain what is meant by an ABI breakage, its
implications and how to deal with them.
ABI (Application Binary Interface) is a term used to describe the
characteristics of binary (i.e. object-) code when accessed from other binary
code. Generally, this entails everything from method signatures over
structure sizes to global data (exported symbols) sizes.
One example of ABI-breakage, if you're somewhat proficient in C, could be the
following:
-- Old code (library header)
/* Some structure which contains a long, and so has total size four bytes on
i386. */
struct x {
long y;
} __attribute__((packed))__;
/* Get access to two x objects. This function is implemented in a shared
library. It returns a pointer to eight bytes of memory (2*struct x). */
extern const struct x* getSomeData();
-- Old code (user)
long test() {
const struct x* ref = getSomeData();
return ref[0].y + ref[1].y;
}
--
Now, when compiling the user code, it will pick up the specification of the
structure x through the library supplied header file, seeing that ref[...].y
is a long value (four bytes on i386), and so that will be translated to some
machine code which adds four bytes from offset zero (ref[0].y) of the pointer
that getSomeData() returns to four bytes from offset four (ref[1].y), and
returns the result as a long value.
What happens if the following change to the library is made?
-- New code (library header)
/* Some structure which now contains a short. This reduces the size of the
structure to two bytes on i386. */
struct x {
short y;
} __attribute__((packed))__;
/* Get access to two x objects. This function is implemented in a shared
library. Due to the change in struct x, this now returns a pointer to (only!)
four bytes of storage (2*struct x). */
extern const struct x* getSomeData();
--
The size of the structure member y of structure x has now been reduced to two
bytes (because the type was changed from long to short), but the user code
doesn't know this, because it was compiled with the old headers. When
installing the changed shared library, the old user code will load (!) as
before, because the symbol getSomeData() is still exported from the shared
library, but will now erraneously load in total eight bytes from a location
(because that's hardcoded in the machine code produced for the user binary)
where it should only load four bytes from after the incompatible change to
the layout of the structure.
If you were to recompile the user code after installing the updated shared
library, everything would go back to functioning normally, because now the
user code picks up the redefined structure x specification which now says
that it should load only two bytes from offset zero (ref[0].y) and two bytes
from offset two (ref[1].y) of the returned pointer.
The API has not changed, because the user code would still compile properly,
but the ABI has, and as such object code compiled before the change in the
shared library breaks (most often in very mysterious ways).
I don't know exactly what will be changed in the kernel to warrant the heads
up for this specific case, but as was said, the vnode structure is changed
(because data was added to it), and as such the structure specification has
changed. If you now have a KLM which expects the old structure layout
(because it was compiled before the change), you're bound for trouble,
because it does not yet know that the offsets for members and the total size
of the vnode structure has changed, and as such all offsets it uses in the
vnode structure are broken.
So, if you have a binary only kernel module which requires access/uses the
vnode structure, you'll have a problem. If not, you don't.
Does this help?
--
Heiko Wundram
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Wednesday 30 July 2008 03:11:54 Heiko Wundram wrote:
Am Mittwoch, 30. Juli 2008 11:47:34 schrieb David Southwell:
For those of us who are not as well informed and experienced as others
could someone please explain what is meant by an ABI breakage, its
implications and how to deal with them.
ABI (Application Binary Interface) is a term used to describe the
characteristics of binary (i.e. object-) code when accessed from other
binary code. Generally, this entails everything from method signatures over
structure sizes to global data (exported symbols) sizes.
One example of ABI-breakage, if you're somewhat proficient in C, could be
the following:
-- Old code (library header)
/* Some structure which contains a long, and so has total size four bytes
on i386. */
struct x {
long y;
} __attribute__((packed))__;
/* Get access to two x objects. This function is implemented in a shared
library. It returns a pointer to eight bytes of memory (2*struct x). */
extern const struct x* getSomeData();
-- Old code (user)
long test() {
const struct x* ref = getSomeData();
return ref[0].y + ref[1].y;
}
--
Now, when compiling the user code, it will pick up the specification of the
structure x through the library supplied header file, seeing that
ref[...].y is a long value (four bytes on i386), and so that will be
translated to some machine code which adds four bytes from offset zero
(ref[0].y) of the pointer that getSomeData() returns to four bytes from
offset four (ref[1].y), and returns the result as a long value.
What happens if the following change to the library is made?
-- New code (library header)
/* Some structure which now contains a short. This reduces the size of the
structure to two bytes on i386. */
struct x {
short y;
} __attribute__((packed))__;
/* Get access to two x objects. This function is implemented in a shared
library. Due to the change in struct x, this now returns a pointer to
(only!) four bytes of storage (2*struct x). */
extern const struct x* getSomeData();
--
The size of the structure member y of structure x has now been reduced to
two bytes (because the type was changed from long to short), but the user
code doesn't know this, because it was compiled with the old headers. When
installing the changed shared library, the old user code will load (!) as
before, because the symbol getSomeData() is still exported from the shared
library, but will now erraneously load in total eight bytes from a location
(because that's hardcoded in the machine code produced for the user binary)
where it should only load four bytes from after the incompatible change to
the layout of the structure.
If you were to recompile the user code after installing the updated shared
library, everything would go back to functioning normally, because now the
user code picks up the redefined structure x specification which now says
that it should load only two bytes from offset zero (ref[0].y) and two
bytes from offset two (ref[1].y) of the returned pointer.
The API has not changed, because the user code would still compile
properly, but the ABI has, and as such object code compiled before the
change in the shared library breaks (most often in very mysterious ways).
I don't know exactly what will be changed in the kernel to warrant the
heads up for this specific case, but as was said, the vnode structure is
changed (because data was added to it), and as such the structure
specification has changed. If you now have a KLM which expects the old
structure layout (because it was compiled before the change), you're bound
for trouble, because it does not yet know that the offsets for members and
the total size of the vnode structure has changed, and as such all offsets
it uses in the vnode structure are broken.
So, if you have a binary only kernel module which requires access/uses the
vnode structure, you'll have a problem. If not, you don't.
Does this help?
It sure helps the understanding bit but how about specific instructions about
who is to what to do in this instance?
i.e. Circumstances and what commands to apply in those circumstances?
The general helps understanding.. the specifics provide solutions for the
problems!!
Thanks again
David
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
Am Mittwoch, 30. Juli 2008 13:11:31 schrieb David Southwell: i.e. Circumstances and what commands to apply in those circumstances? If you don't know what to do, don't run -STABLE? -- Heiko Wundram ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Wed, 30 Jul 2008 13:02:52 +0200, Heiko Wundram [EMAIL PROTECTED] wrote: Am Mittwoch, 30. Juli 2008 13:11:31 schrieb David Southwell: i.e. Circumstances and what commands to apply in those circumstances? If you don't know what to do, don't run -STABLE? I think in this case (the ABI breakage) it is more nice to say: If you don't know what to do, you wil probably not see any problem because of this. The edge case of what can go wrong is so small, that you must be doing quite specialized stuff to see breakage. In that case you would understand what is going on. (IMHO) Ronald. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
Am Mittwoch, 30. Juli 2008 13:03:34 schrieb Ronald Klop: I think in this case (the ABI breakage) it is more nice to say: If you don't know what to do, you wil probably not see any problem because of this. The edge case of what can go wrong is so small, that you must be doing quite specialized stuff to see breakage. In that case you would understand what is going on. (IMHO) Err, no. As someone else has already noted, a prominent KLM that's distributed separately from the kernel which uses the vnode structure extensively (which I didn't think of at all until I read the respective mail) is fuse. A pre-upgrade compiled fuse is most certainly going to break because of this change. Some people have dual installations of Windows/FreeBSD (at least I'd presume that's the case with the fiddlers like me that track -STABLE as a hobby, not as a developer, or those developers who program for Windows as a day-job, also like me) who use ntfs-3g to mount their NTFS-data; additionally, I also extensively use ssh-fs, which is also fuse-based, and I guess there are also others who use it, and so the reach of this ABI-change, at least IMHO, is much larger than the original message makes you believe. Now, after the update, a lot can go wrong, because the fuse KLM is loaded by an init-script, and your system is most probably going to Oops while booting if you didn't think of disabling the fuse init-script before you update your kernel, and will NOT fail gracefully. If the respective person doesn't know how he/she should boot to single-user-mode, update rc.conf to disable this, reboot, rebuild the module to get the system back up, the only thing I can possibly say is: don't track stable. It might've sounded a bit harsh what I wrote, but tracking -STABLE means knowing your system enough so that you know how to fix things if they come back to bite you (especially after getting a HEADS UP). And that doesn't seem to be the case here if the respective person asks for SPECIFIC instructions what to do. So, again: DON'T track -STABLE if you can't fix the system if it breaks, and AFAICT this change is most certainly going to break quite a few systems. -- Heiko Wundram ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Wed, Jul 30, 2008 at 01:26:33PM +0200, Heiko Wundram wrote: Am Mittwoch, 30. Juli 2008 13:03:34 schrieb Ronald Klop: I think in this case (the ABI breakage) it is more nice to say: If you don't know what to do, you wil probably not see any problem because of this. The edge case of what can go wrong is so small, that you must be doing quite specialized stuff to see breakage. In that case you would understand what is going on. (IMHO) Err, no. As someone else has already noted, a prominent KLM that's distributed separately from the kernel which uses the vnode structure extensively (which I didn't think of at all until I read the respective mail) is fuse. A pre-upgrade compiled fuse is most certainly going to break because of this change. Some people have dual installations of Windows/FreeBSD (at least I'd presume that's the case with the fiddlers like me that track -STABLE as a hobby, not as a developer, or those developers who program for Windows as a day-job, also like me) who use ntfs-3g to mount their NTFS-data; additionally, I also extensively use ssh-fs, which is also fuse-based, and I guess there are also others who use it, and so the reach of this ABI-change, at least IMHO, is much larger than the original message makes you believe. Now, after the update, a lot can go wrong, because the fuse KLM is loaded by an init-script, and your system is most probably going to Oops while booting if you didn't think of disabling the fuse init-script before you update your kernel, and will NOT fail gracefully. If the respective person doesn't know how he/she should boot to single-user-mode, update rc.conf to disable this, reboot, rebuild the module to get the system back up, the only thing I can possibly say is: don't track stable. It might've sounded a bit harsh what I wrote, but tracking -STABLE means knowing your system enough so that you know how to fix things if they come back to bite you (especially after getting a HEADS UP). And that doesn't seem to be the case here if the respective person asks for SPECIFIC instructions what to do. So, again: DON'T track -STABLE if you can't fix the system if it breaks, and AFAICT this change is most certainly going to break quite a few systems. I do not use the fuse fs myself. But the issue was raised during the internal discussion of the MFC, and it seems that fuse could be not affected. The change only adds the field at the end of the vnode structure, so all existing fields offsets are intact. pgp9l7kiqptz3.pgp Description: PGP signature
Re: Upcoming ABI Breakage in RELENG_7
On Wednesday 30 July 2008 04:26:33 Heiko Wundram wrote: Am Mittwoch, 30. Juli 2008 13:03:34 schrieb Ronald Klop: I think in this case (the ABI breakage) it is more nice to say: If you don't know what to do, you wil probably not see any problem because of this. The edge case of what can go wrong is so small, that you must be doing quite specialized stuff to see breakage. In that case you would understand what is going on. (IMHO) Err, no. As someone else has already noted, a prominent KLM that's distributed separately from the kernel which uses the vnode structure extensively (which I didn't think of at all until I read the respective mail) is fuse. A pre-upgrade compiled fuse is most certainly going to break because of this change. Some people have dual installations of Windows/FreeBSD (at least I'd presume that's the case with the fiddlers like me that track -STABLE as a hobby, not as a developer, or those developers who program for Windows as a day-job, also like me) who use ntfs-3g to mount their NTFS-data; additionally, I also extensively use ssh-fs, which is also fuse-based, and I guess there are also others who use it, and so the reach of this ABI-change, at least IMHO, is much larger than the original message makes you believe. Now, after the update, a lot can go wrong, because the fuse KLM is loaded by an init-script, and your system is most probably going to Oops while booting if you didn't think of disabling the fuse init-script before you update your kernel, and will NOT fail gracefully. If the respective person doesn't know how he/she should boot to single-user-mode, update rc.conf to disable this, reboot, rebuild the module to get the system back up, the only thing I can possibly say is: don't track stable. It might've sounded a bit harsh what I wrote, but tracking -STABLE means knowing your system enough so that you know how to fix things if they come back to bite you (especially after getting a HEADS UP). And that doesn't seem to be the case here if the respective person asks for SPECIFIC instructions what to do. So, again: DON'T track -STABLE if you can't fix the system if it breaks, and AFAICT this change is most certainly going to break quite a few systems. Hey I want to thank those that have taken the trouble to explain stuff BUT I also feel the need to object to If you cant then don't kind of mentality. Everyone has to start somewhere!!! A contribution that sounds a bit like saying: Well if you do not have the experience on tracking stable then do not bother to start OR If you do not understand my jargon why should it be explained seems to me to invite a gentle reproof David ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
David Southwell wrote: On Tuesday 29 July 2008 08:45:45 Ken Smith wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be Sometimes information gets posted to this list on the assumption that everyone understand what the writer means. This is one of those occasions!! For those of us who are not as well informed and experienced as others could someone please explain what is meant by an ABI breakage, its implications and how to deal with them. Within a major release, the project tries very hard to maintain Binary Interface campatibility, or stability. In fact as far as I know, this is where the name Stable Branch originates. What this means is that a binary compiled on 7.0-RELEASE should continue to work without needing to be recompiled over the lifetime of the 7 branch. ABI breakage that Ken refers to here means that a change required to fix a bug cannot be made while maintaining binary compatibility with previous versions. Any program that makes use of (I'm guessing) fcntl(2) or flock(2) that runs on 7.0, will not run on = 7.1 without being recompiled. Ian -- Ian Freislich ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
On Wed, 2008-07-30 at 07:55 -0700, David Southwell wrote: On Wednesday 30 July 2008 07:27:27 Ian FREISLICH wrote: David Southwell wrote: On Tuesday 29 July 2008 08:45:45 Ken Smith wrote: Normally the FreeBSD Project tries very hard to avoid ABI breakage in Stable Branches. However occasionally the fix for a bug can not be Sometimes information gets posted to this list on the assumption that everyone understand what the writer means. This is one of those occasions!! For those of us who are not as well informed and experienced as others could someone please explain what is meant by an ABI breakage, its implications and how to deal with them. Within a major release, the project tries very hard to maintain Binary Interface campatibility, or stability. In fact as far as I know, this is where the name Stable Branch originates. What this means is that a binary compiled on 7.0-RELEASE should continue to work without needing to be recompiled over the lifetime of the 7 branch. ABI breakage that Ken refers to here means that a change required to fix a bug cannot be made while maintaining binary compatibility with previous versions. Any program that makes use of (I'm guessing) fcntl(2) or flock(2) that runs on 7.0, will not run on = 7.1 without being recompiled. Actually not quite, and that's why Konstantin mentioned I made a slight mistake in my original announcement. ABI stands for *Application* Binary Interface but (sorry) that's not quite what is being broken here. It is a KBI (Kernel Binary Interface) that's being broken. The difference is that the data structures being changed are strictly internal to the kernel. Back in the days of strictly monolithic kernels when the only way to get something into the kernel was to rebuild the whole kernel and reboot with it this sort of thing was less of a concern. But now that kernel modules can be loaded while the machine is up and running we need to be a bit more careful about announcing when a very important kernel data structure gets modified in such a way that already-compiled modules might not be compatible because of, as someone else mentioned, that very important data structure changing in size or the elements inside that structure changing position. So because this is a KBI being broken it should NOT have any impact on user-level executable files. It should only impact loadable kernel modules that deal with filesystems. IF this change did indeed change user-level programs to the degree mentioned above that would raise the bar *dramatically* on whether we would consider the fix worth the fall-out that would occur. Frankly I think we would have rejected the fix if it were that drastic. Thanks that is very concise and very helpful. Question is what is the best way to determine which programs (if any) depend upon fcntl or flock on a running system? No need to. :-) But one way to do it if you ever needed to is with nm(1), something like: nm file | grep fcntl though that would only tell you if that executable directly calls fcntl(2) itself (functions inside libraries may wind up calling fcntl(2) so if you really needed to be thorough you would need to also check the libraries the executable is linked against, ldd(1) can help you figure out what those are). There are likely better/easier ways to do it. -- Ken Smith - From there to here, from here to | [EMAIL PROTECTED] there, funny things are everywhere. | - Theodore Geisel | signature.asc Description: This is a digitally signed message part
Re: Upcoming ABI Breakage in RELENG_7
David Southwell wrote: On Wednesday 30 July 2008 07:27:27 Ian FREISLICH wrote: ABI breakage that Ken refers to here means that a change required to fix a bug cannot be made while maintaining binary compatibility with previous versions. Any program that makes use of (I'm guessing) fcntl(2) or flock(2) that runs on 7.0, will not run on = 7.1 without being recompiled. Thanks that is very concise and very helpful. Question is what is the best way to determine which programs (if any) depend upon fcntl or flock on a running system? That will be just about anything that opens and locks files. RDBMS, editors, browsers, MTAs spring to mind. This of course depends on my initial guess being correct. Ian -- Ian Freislich ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
* Ian FREISLICH [EMAIL PROTECTED] wrote: ABI breakage that Ken refers to here means that a change required to fix a bug cannot be made while maintaining binary compatibility with previous versions. Any program that makes use of (I'm guessing) fcntl(2) or flock(2) that runs on 7.0, will not run on = 7.1 without being recompiled. Are you sure? I fcntl(2) and flock(2) will not change. Only some kernel space bits will. -- Ed Schouten [EMAIL PROTECTED] WWW: http://80386.nl/ pgponVKKhfTne.pgp Description: PGP signature
Re: Upcoming ABI Breakage in RELENG_7
David Southwell wrote: It sure helps the understanding bit but how about specific instructions about who is to what to do in this instance? I fear that the forest is getting lost for the trees here. :) Under most (90% or more) circumstances stuff, be that kernel modules or userland binaries, that is on a -stable system will still work in its current form after you do an upgrade. In the event of an actual ABI breakage you would be informed of when and where to tune for additional wait, wrong PSA. If this had actually been an ABI breakage then any userland binaries that depend on the thing that changed would have to be recompiled after the update in order to continue working. Since this is actually a KERNEL interface that's changing, any kernel modules that use this interface (which is quite a lot) will have to be recompiled _as part of the update_ for them to work. If you are not using any 3rd party kernel modules (commercial, ports tree, etc.) then you have nothing to worry about. Your FreeBSD kernel modules will be updated along with the kernel, and everyone is happy. If you are using a 3rd party module from ports that deals with file systems (and yes, FUSE is an obvious example here) then you'll need to pkg_delete it, do the update, then recompile. Q.E.D. If you happen to be using a commercial (i.e., you don't have the source) kernel module that deals with file systems now is the time to contact your vendor and let them know about this issue. However, as Ken rightly pointed out this change will actually impact a very small percentage of FreeBSD users, and the majority of that impact will be recompiling a port as described above. I should also point out that FreeBSD developers deal with this issue all the time on -current since things there are allowed to change, and sometimes do change quite often. Therefore it's easy for us to forget to list important details about a change like this since it's second nature to us. hth, Doug -- This .signature sanitized for your protection ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Upcoming ABI Breakage in RELENG_7
Am Mittwoch, 30. Juli 2008 13:30:31 schrieben Sie: I do not use the fuse fs myself. But the issue was raised during the internal discussion of the MFC, and it seems that fuse could be not affected. The change only adds the field at the end of the vnode structure, so all existing fields offsets are intact. I just half-throughly waded through the sysutils/fusefs-kmod sources, and you're right; there's no mentioning of sizeof(struct vnode) in all of the preprocessed sources (at least none that I could find, directly or indirectly). So, if struct vnode only grows and none of the original fields are moved due to padding issues (or gcc reordering fields because of padding), the old kernel module should work unchanged. To finish: sorry for the noise, and sorry for the panic I might've provoked. -- Heiko Wundram ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
