Re: ABI Compatibility (was Re: Installation woes for the average user...)
Richard Frith-Macdonald wrote: Chasing MacOS-X is a problem for us .. sometimes Apple make changes which require ABI changes. Often they add new stuff, which we try to add too. If we don't get the additions quite right in one release, we need to fix them in the next release, and that fix may change the ABI with respect to the new feature, even though the ABI of the main part of the library is stable/unaffected. This can be something as trivial as implementing an obscure method with an argument of the wrong type (eg a long rather than an int), then having to change the argument type in a later release to match MacOS-X. I find it quite funny that you regularly bring up Mac OS X compatibility as a reason for ABI breakage. Have you ever looked at what Apple is doing? I can use executables compiled on Mac OS X 10.2 on Mac OS X 10.5 without any problems and Mac OS X does ship with only a single instance of all its libraries/frameworks. I.e., it was possible for Apple to keep the ABI stable and avoid any soname changes for all major libraries since at least 10.2. For sure, they play some tricks here and check at runtime whether an executable was linked against the 10.2, 10.3, 10.4, or 10.5 frameworks and adapt the behavior of some methods accordingly. Wolfgang ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 21 May 2009, at 10:34, Wolfgang Lux wrote: Have you ever looked at what Apple is doing? I can use executables compiled on Mac OS X 10.2 on Mac OS X 10.5 without any problems and Mac OS X does ship with only a single instance of all its libraries/ frameworks Only half of this is true. Apple only ships one version of its frameworks. Each framework, however, contains different versions of the libraries. You may have noticed all of the OS X version macros in the headers. Each of those defines the parts of the interface that are visible in a particular version of the framework, but they are also used when compiling. Each version of OS X has a Foundation.framework containing several libFoundation.dylib files, one compiled for compatibility with each previous version. Without access to their source, I can't be completely sure how they are doing this, but my guess is that each ivar and each method - even each typedef - has the same feature macros in front of it and their build system automatically generates a backwards-compatible version as long as these changes are made. It would be nice if GNUstep make had facilities for doing this, but I imagine it would be a lot of effort. It would also be easier if we could rely on a loader that supported frameworks properly, but I believe Matt Rice is the only person who has one of these on a non- NeXT/Apple platform. David ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
David Chisnall wrote: On 21 May 2009, at 10:34, Wolfgang Lux wrote: Have you ever looked at what Apple is doing? I can use executables compiled on Mac OS X 10.2 on Mac OS X 10.5 without any problems and Mac OS X does ship with only a single instance of all its libraries/frameworks Only half of this is true. Apple only ships one version of its frameworks. Each framework, however, contains different versions of the libraries. You may have noticed all of the OS X version macros in the headers. Each of those defines the parts of the interface that are visible in a particular version of the framework, but they are also used when compiling. Each version of OS X has a Foundation.framework containing several libFoundation.dylib files, one compiled for compatibility with each previous version. Without access to their source, I can't be completely sure how they are doing this, but my guess is that each ivar and each method - even each typedef - has the same feature macros in front of it and their build system automatically generates a backwards-compatible version as long as these changes are made. I'm sorry, but this is just plain wrong. There is just one Foundation, AppKit, etc. library on each system. Maybe you are thinking of the libraries in the /Developer/SDKs/MacOSX10xxx.sdk directories, but these are just stub libraries that are provided in order to allow *linking* of executables for another OS version. These stub libraries just contain the necessary symbol definitions but no code at all. It would be nice if GNUstep make had facilities for doing this, but I imagine it would be a lot of effort. It would also be easier if we could rely on a loader that supported frameworks properly, but I believe Matt Rice is the only person who has one of these on a non- NeXT/Apple platform. I don't think that frameworks are the issue here. After all, frameworks are just a convenient way to group headers, libraries, and resources of a particular project together. Sure, they also provide for versioning, but Apple does not seem to use that, e.g., Apple's Foundation and AppKit libraries are both at version C since the advent of Mac OS X. Wolfgang ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Richard Frith-Macdonald wrote: On 21 May 2009, at 10:34, Wolfgang Lux wrote: Richard Frith-Macdonald wrote: Chasing MacOS-X is a problem for us .. sometimes Apple make changes which require ABI changes. Often they add new stuff, which we try to add too. If we don't get the additions quite right in one release, we need to fix them in the next release, and that fix may change the ABI with respect to the new feature, even though the ABI of the main part of the library is stable/ unaffected. This can be something as trivial as implementing an obscure method with an argument of the wrong type (eg a long rather than an int), then having to change the argument type in a later release to match MacOS-X. I find it quite funny that you regularly bring up Mac OS X compatibility as a reason for ABI breakage. Have you ever looked at what Apple is doing? I can use executables compiled on Mac OS X 10.2 on Mac OS X 10.5 without any problems and Mac OS X does ship with only a single instance of all its libraries/frameworks. I.e., it was possible for Apple to keep the ABI stable and avoid any soname changes for all major libraries since at least 10.2. For sure, they play some tricks here and check at runtime whether an executable was linked against the 10.2, 10.3, 10.4, or 10.5 frameworks and adapt the behavior of some methods accordingly. Yes, with the manpower they have (and perhaps more importantly, the fact that they control the entire operating system) it's relatively easy to add runtime tricks to pick the behaviors they want. I don't think that it is *that* difficult. I guess that for determining the OS version an executable was linked on they use features of the mach-o file format, namely that the exact version of a dynamic library is available in the file's header. AFAICT this is not true for the elf format, but one could easily emulate this behavior, e.g., by accompanying each dynamic library with a small object file that exports only the build version of the library and is linked statically with an executable using that library. Also oftentimes they can just get away with doing things (like the transition from int to NSIntegerbreaking API for 64bit systems) because they simply didn't support things before, so there's no old behavior to be broken. Wolfgang ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 17 Mar 2009, at 10:28, Yavor Doganov wrote: This is probably worth having in the (very) distant future, but has little to do with the question at hand. My objection was that it's absolutely useless to bump the soname 1.14 - 1.15 - 1.16 (just an example) when there are only compatible bugfixes and API additions. How you version the releases and what is considered stable and unstable is completely orthogonal to the library versioning. This was the last post on this subject, but apparently the point was not absorbed by anyone. Why did the library version number bump when I updated GNUstep? This left all of my apps and frameworks linked to the old version (which, because the new version had a different name, wasn't overwritten) even though the ABI was compatible. Let me repeat that: This policy caused every framework and every application to require recompiling (well, technically only relinking, but good luck persuading GNUstep make to do that) FOR NO REASON. This is exactly the kind of thing that makes distributions reluctant to ship up-to-date GNUstep. Please, please, please, stop it. The soname should only be bumped when the ABI changes in an incompatible way. David ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 20 May 2009, at 10:34, David Chisnall wrote: On 17 Mar 2009, at 10:28, Yavor Doganov wrote: This is probably worth having in the (very) distant future, but has little to do with the question at hand. My objection was that it's absolutely useless to bump the soname 1.14 - 1.15 - 1.16 (just an example) when there are only compatible bugfixes and API additions. How you version the releases and what is considered stable and unstable is completely orthogonal to the library versioning. This was the last post on this subject, but apparently the point was not absorbed by anyone. Why did the library version number bump when I updated GNUstep? This left all of my apps and frameworks linked to the old version (which, because the new version had a different name, wasn't overwritten) even though the ABI was compatible. Let me repeat that: This policy caused every framework and every application to require recompiling (well, technically only relinking, but good luck persuading GNUstep make to do that) FOR NO REASON. This is exactly the kind of thing that makes distributions reluctant to ship up-to-date GNUstep. Please, please, please, stop it. The soname should only be bumped when the ABI changes in an incompatible way. I think what you are describing (as the correct way to do things) is what GNUstep already tries to do ... see http://mediawiki.gnustep.org/index.php/GNUstep_release_policy for the actual release policy details. While this: How you version the releases and what is considered stable and unstable is completely orthogonal to the library versioning. is technically true, it's not what people expect in practice, and therefor not policy. The policy is to keep the main so version numbers in sync with the main release version numbers, because that matches normal expectations. However, I don't think there have been any recent releases which should have changed the so number for any of the libraries. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On Wed, May 20, 2009 at 11:24:39AM +0100, Richard Frith-Macdonald wrote: I think what you are describing (as the correct way to do things) is what GNUstep already tries to do ... see http://mediawiki.gnustep.org/index.php/GNUstep_release_policy for the actual release policy details. On the contrary -- this very policy is the problem. The SONAME of a shared library should be bumped if and only if there is ABI break, not because of publicity reasons, big API additions, or anything else. You can continue to version stable releases with even numbers and unstable releases with odd numbers and release them in any order you feel apropriate; the SONAME has absolutely nothing to do with the package version. How you version the releases and what is considered stable and unstable is completely orthogonal to the library versioning. is technically true, it's not what people expect in practice, and therefor not policy. The policy is to keep the main so version numbers in sync with the main release version numbers, because that matches normal expectations. Name just one library -- just one -- on your GNU system that behaves this way. Consider what would happen if glibc or glib/gtk+ bump the soname every six months... I find it incredibly hard to believe that this is normal expectation. However, I don't think there have been any recent releases which should have changed the so number for any of the libraries. I've made an experiment some time ago by altering the symlinks without recompiling anything. Every GNUstep app in Debian worked out of the box (of course that's not a 100% proof that the new Base/GUI were ABI compatible). ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 20 May 2009, at 11:53, Yavor Doganov wrote: On Wed, May 20, 2009 at 11:24:39AM +0100, Richard Frith-Macdonald wrote: I think what you are describing (as the correct way to do things) is what GNUstep already tries to do ... see http://mediawiki.gnustep.org/index.php/GNUstep_release_policy for the actual release policy details. On the contrary -- this very policy is the problem. The SONAME of a shared library should be bumped if and only if there is ABI break, not because of publicity reasons, big API additions, or anything else. That's a matter of opinion. The GNUstep policy reserves the right to change the soname solely for publicity reasons, but afaik has never done so. In practice pretty much all soname changes are related to ABI changes. You can continue to version stable releases with even numbers and unstable releases with odd numbers and release them in any order you feel apropriate; the SONAME has absolutely nothing to do with the package version. How you version the releases and what is considered stable and unstable is completely orthogonal to the library versioning. is technically true, it's not what people expect in practice, and therefor not policy. The policy is to keep the main so version numbers in sync with the main release version numbers, because that matches normal expectations. Name just one library -- just one -- on your GNU system that behaves this way. Hmm ... looks like you are right on that one ... I remember that user expectations were an argument years ago, back when the policy was originally formed, but perhaps that's changed (or perhaps it was never valid, just assumed). Consider what would happen if glibc or glib/gtk+ bump the soname every six months... I find it incredibly hard to believe that this is normal expectation. Er ... frequency of release of binary incompatible code is a completely different issue from version numbering. The fact is, most GNUstep libraries are still having to change. Even the base library needs to break binary compatibility occasionally in order to improve MacOS-X compatibility. However, I don't think there have been any recent releases which should have changed the so number for any of the libraries. I've made an experiment some time ago by altering the symlinks without recompiling anything. Every GNUstep app in Debian worked out of the box (of course that's not a 100% proof that the new Base/GUI were ABI compatible). I agree ... not even close to proof. Most ABI changes are minor and won't effect most applications ... but we still need to change the soname because we don't know whether there are apps out there which will break. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 20 May 2009, at 12:29, David Chisnall wrote: In fact, this link appears to directly contradict itself: The library (SONAME) versions is changed when the major or minor version number of a release changes, but not the subminor number. and The minor version number is changed (and therefore the library version) when we break backward compatibility In what way does that contradict itsself? major number changed implies soname changed minor number changed implies soname changed subminor number changed does not imply soname changed backward compatibility broken, implies minor number changed, which implies soname changed This does not appear to be what really happens. I guess appearances can be deceptive. The minor version number is changed with each new release. Not so. For example, we have base 1.19.0 and 1.19.1 and we have base 1.16.0 and 1.16.1 and 1.16.3 and 1.16.4 and 1.16.5 I suppose there is a tendency to only notice when new releases have new soname versions, and not notice when they don't. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Richard Frith-Macdonald wrote: On 20 May 2009, at 11:53, Yavor Doganov wrote: On Wed, May 20, 2009 at 11:24:39AM +0100, Richard Frith-Macdonald wrote: However, I don't think there have been any recent releases which should have changed the so number for any of the libraries. I've made an experiment some time ago by altering the symlinks without recompiling anything. Every GNUstep app in Debian worked out of the box (of course that's not a 100% proof that the new Base/GUI were ABI compatible). I agree ... not even close to proof. Most ABI changes are minor and won't effect most applications ... but we still need to change the soname because we don't know whether there are apps out there which will break. I know that there have been incompatible changes to the gui library, I am very sure about that as I made them. You are right, though. Most applications wouldn't have noticed and for base the change probably wasn't needed. Fred ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Richard Frith-Macdonald wrote: On 20 May 2009, at 11:53, Yavor Doganov wrote: The SONAME of a shared library should be bumped if and only if there is ABI break That's a matter of opinion. Yes, the opinion of the library maintainer(s), which ideally should conform to best practices. The worst thing to do is to keep the same SONAME when there's an ABI break. Gratuitously bumping the SONAME is not so bad, but it's a *big* annoyance to users and large binary distributions who have to deal with library transitions, sometimes inter-related and entangled in complicated ways. The fact is, most GNUstep libraries are still having to change. No problem here. Even the base library needs to break binary compatibility occasionally in order to improve MacOS-X compatibility. Occasionally, you say. If I understand correctly, you say that between 1.13 and 1.19 (for example) not all soname bumps were warranted. If so, we agree here, and that's the core of the problem. Anyway, I am not qualified enough to argue with you -- someone with enough spare time, skills and affinity to archaelogy could eventually examine the diffs and prove/disprove the claim. Most ABI changes are minor and won't effect most applications ... but we still need to change the soname because we don't know whether there are apps out there which will break. That's the right thing to do, indeed, iff there is ABI breakage. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 20 May 2009, at 20:54, Yavor Doganov wrote: Richard Frith-Macdonald wrote: Even the base library needs to break binary compatibility occasionally in order to improve MacOS-X compatibility. Occasionally, you say. If I understand correctly, you say that between 1.13 and 1.19 (for example) not all soname bumps were warranted. No that's absolutely *not* what I said. I pointed out that the large majority of the recent base library releases (including the latest one) did *not* change the soname! I expect the next one on the unstable branch probably will need to change the soname, because it is likely that it will adopt the new MacOS-X 10.5 NSInteger and NSUinteger etc behaviors for 64bit processors (currently you have to turn that compatibility on) which will break binary compatibility on 64bit machines. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 20 May 2009, at 23:17, Yavor Doganov wrote: Richard Frith-Macdonald wrote: On 20 May 2009, at 20:54, Yavor Doganov wrote: If I understand correctly, you say that between 1.13 and 1.19 (for example) not all soname bumps were warranted. No that's absolutely *not* what I said. I pointed out that the large majority of the recent base library releases (including the latest one) did *not* change the soname! I feel we are miscommunicating. Let me try to clear this out -- I am counting only the major releases when there was a soname bump; the regular point releases are not interesting for this discussion. Then the discussion seems rather pointless ... since what you are calling the 'major' releases are those were we make ABI changes and bump the soname. 1.13.0 was released on 2006-08-28 according to the ChangeLog entry, so for less than 3 years you put your word that you broke the ABI 6 times? That's astonishing given the fact that Base is advertised as a fairly stable library (and *it is*, more or less). Yes, there's a *lot* of functionality there, and the core part of it is very stable, so the chances are that any individual ABI change will not break your application because the app will not be using the feature concerned. In particular, changes to class instance variable layouts will only effect applications which subclass the class concerned ... and that's very, very few applications for most classes. Chasing MacOS-X is a problem for us .. sometimes Apple make changes which require ABI changes. Often they add new stuff, which we try to add too. If we don't get the additions quite right in one release, we need to fix them in the next release, and that fix may change the ABI with respect to the new feature, even though the ABI of the main part of the library is stable/unaffected. This can be something as trivial as implementing an obscure method with an argument of the wrong type (eg a long rather than an int), then having to change the argument type in a later release to match MacOS-X. Given that, I think it's more likely that we sometimes make a mistake and release without bumping the soname when we technically should, than that we accidentally bump the soname when we don't need to. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
В Fri, 13 Mar 2009 13:00:19 -0600, Adam Fedor написа: We had a long discussion about this before Yes, I remember it. that ended in the request that stable releases have no ABI or API changes (even additions), so that a developer and user could count on the same set of functionality in a release. This is probably worth having in the (very) distant future, but has little to do with the question at hand. My objection was that it's absolutely useless to bump the soname 1.14 - 1.15 - 1.16 (just an example) when there are only compatible bugfixes and API additions. How you version the releases and what is considered stable and unstable is completely orthogonal to the library versioning. In fact, Glib/GTK+ has a similar release version numbering and development routine as GNUstep. Odd versions are unstable, even versions are stable, but they never break the ABI because they promised so. It shouldn't be very hard if GNUstep libraries break the ABI only when the developers actually do it, and not always with every major release. Notice the difference from the super-stability that Helge was talking about in that old thread. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On Mar 12, 2009, at 2:19 AM, Yavor Doganov wrote: I think you misunderstood. As a GNUstep user and distro maintainer of GNUstep packages, I don't mind if the GNUstep developers break the ABI as much as they want (of course, the less the better). But only bump the soname of a library when there is breakage. The current release policy just doesn't make any sense (and never had for me personally). We had a long discussion about this before that ended in the request that stable releases have no ABI or API changes (even additions), so that a developer and user could count on the same set of functionality in a release. It's a nice goal, but GNUstep really isn't in a place where it is stable enough for this. http://www.mail-archive.com/gnustep-dev@gnu.org/msg03452.html ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
В Wed, 11 Mar 2009 12:53:56 +, David Chisnall написа: It is very easy to have a stable ABI for a procedural API because there is no notion of subclassing. Even with GObject this is not really a problem, as it doesn't really present an OO model. Right you are. Even then, after years of deprecating functions/entire classes and adding new fancy stuff, the GTK+ developers are considering to break the API/ABI with 3.0. It is not unusual for a large library to become unmaintainable at some point, accumulating unthinkable amounts of cruft, and the need for refactoring can become obvious. It would be good enough to break the ABI *only* when necessary. I'd agree with this. At the very least, I'd hope that people would post a mail to gnustep-dev BEFORE breaking the ABI and not commit I think you misunderstood. As a GNUstep user and distro maintainer of GNUstep packages, I don't mind if the GNUstep developers break the ABI as much as they want (of course, the less the better). But only bump the soname of a library when there is breakage. The current release policy just doesn't make any sense (and never had for me personally). ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Hello Xavier,
Am Dienstag, den 10.03.2009, 17:49 +0100 schrieb Xavier Glattard:
I'm sorry, i still can't see any problem.
NSAllocateObject is implemented as this : (NSObject.m:767)
size = aClass-instance_size + extraBytes + sizeof(struct obj_layout);
new = NSZoneMalloc(zone, size);
Then the extra bytes are allocated always _after_ the class ivar,
whatever the class is : the parent class or a subclass. If you get
isa-instance_size (as *you* did) you find the extra bytes.
It is true that the overall size of the instance is correct...
parent class ayout:
{
{ // the instance ivar
int a;
}
{ // the extra ivar
int a_ext;
}
}
subclass layout:
{
{ // the instance ivars
int a;
int b;
}
{ // the extra ivars
int a_ext;
}
}
This is the same behavior than 'classic' external ivars, but the memory
is allocated along with the instance itself (remember: the question is
'no extra malloc call'). The pointer to external ivars is not stored as
is, but is computed with self and instance_size. Still an 'extra load',
but not more than with a compiler-side solution. I even think this is a
'hand made' non-fragile ivar system, with no need for compiler support.
And the alignment issue seems to be solved with padding in
NSObject.m:334-371
All true but you need consider the assembler code that the compiler
emits to access the ivar in each class. And here the parent and the
subclass will disagree on what the offset to a_ext is.
This offset is currently an offset fixed at compile time by the
compiler's calculation of:
struct {
@defs(ParentClass)
} *obj;
obj[1] /* the address of a_ext */
which cannot take into account the ivars of potential subclasses, which
will be at that exact same address (i.e obj[1] == b).
It takes the non-fragile ivars to replace the fixed value (calculated
and emitted by the compiler) into a lookup to the actual offset/location
of the value. This mechanism can only be provided by a newer compiler
emitting that lookup code instead of using the fixed offsets.
Hope that makes it clear why using the extra data prevents subclasses
from adding ivars.
Cheers,
David
___
Gnustep-dev mailing list
Gnustep-dev@gnu.org
http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
В Mon, 09 Mar 2009 22:23:25 +0100, Riccardo Mottola написа: If we compare to other projects like glib2, gtk2: fine-grained packages like debian will give you udpates only when necessary Glib/GTK+ have a stable API/ABI throughout the whole lifetime of the GNOME 2 platform. If you upgrade from GTK+ 2.8 to GTK+ 2.14 (say), no recompilation of GTK apps is required. This is drastically different from the GNUstep libraries which break the ABI once or twice every year. Of course, API/ABI stability is a big promise, and personally I don't think it's worth the trouble for GNUstep at this point. It would be good enough to break the ABI *only* when necessary. (but this can be done for gnustep too, without problems) The problems with GNUstep, at least for binary distributions, is the regular library transitions with recompilation of all GNUstep packages. Sometimes it's necessary, sometimes it's not, but they are forced to do it anyway. There is zero benefit for the end user when the transition is useless -- instead of updating only Base/GUI/Back, she updates the whole stack of apps/tools/frameworks. On system where you build, like NetBSD with pkgsrc or gentoo, one update trigegrs the recompilation of all packages. Just did that yesterday: update glib2? Be prepared to recompile up to your browser. This seems like a broken system (or more likely, user setup) to me. All glib2 releases are binary compatible, so no recompilation of its reverse dependencies is necessary. It's just like when you upgrade from glibc 2.7 to 2.9 you don't recompile everything -- the ABI is the same; there are only API additions. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 11 Mar 2009, at 08:26, Yavor Doganov wrote: Glib/GTK+ have a stable API/ABI throughout the whole lifetime of the GNOME 2 platform It is very easy to have a stable ABI for a procedural API because there is no notion of subclassing. Even with GObject this is not really a problem, as it doesn't really present an OO model. The GNUstep ABI changes are USUALLY only a problem when an application has subclasses of that class that changed. Some classes are subclassed more often than others. A change to the ivar layout of NSObject would break everything, but this never happens (NSObject has no ivars, and never will). A change to something like NSView would break a lot of things, but there are still a lot of applications with no custom NSView subclasses. Of course, API/ABI stability is a big promise, and personally I don't think it's worth the trouble for GNUstep at this point. It would be good enough to break the ABI *only* when necessary. I'd agree with this. At the very least, I'd hope that people would post a mail to gnustep-dev BEFORE breaking the ABI and not commit until there had been some discussion of whether it really is a problem. Committing to trunk and reverting if enough people complain is not really a good approach. David ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
David Ayers a écrit :
Hello Xavier,
Am Dienstag, den 10.03.2009, 17:49 +0100 schrieb Xavier Glattard:
(...)
This is the same behavior than 'classic' external ivars, but the memory
is allocated along with the instance itself (remember: the question is
'no extra malloc call'). The pointer to external ivars is not stored as
is, but is computed with self and instance_size. Still an 'extra load',
but not more than with a compiler-side solution. I even think this is a
'hand made' non-fragile ivar system, with no need for compiler support.
And the alignment issue seems to be solved with padding in
NSObject.m:334-371
All true but you need consider the assembler code that the compiler
emits to access the ivar in each class. And here the parent and the
subclass will disagree on what the offset to a_ext is.
This offset is currently an offset fixed at compile time by the
compiler's calculation of:
struct {
@defs(ParentClass)
} *obj;
obj[1] /* the address of a_ext */
which cannot take into account the ivars of potential subclasses, which
will be at that exact same address (i.e obj[1] == b).
It takes the non-fragile ivars to replace the fixed value (calculated
and emitted by the compiler) into a lookup to the actual offset/location
of the value. This mechanism can only be provided by a newer compiler
emitting that lookup code instead of using the fixed offsets.
Hope that makes it clear why using the extra data prevents subclasses
from adding ivars.
All this is now very clear for me ;)
At first I forgot that 'self+1' is evaluated by the compiler and then
parent and subclasses get the same value when they should not. But David
Chisnal introduced the 'isa-instance_size' value, that is evaluated at
runtime.
Anyway I agree with many (all?) of you : this is quite an ugly solution,
with some drawbacks.
What about this Obj-C linker that would load the library version needed
by the application?
- Regards
-- Xavier
___
Gnustep-dev mailing list
Gnustep-dev@gnu.org
http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Riccardo Mottola a écrit :
(...)
David Chisnall wrote:
(...)
Yes, this appears to me the least hurting path. But I laready dislike.
If we really need, this should be the road.
If you make private ivars into a structure and make a pointer to this
an ivar, you add an extra malloc for every +alloc (expensive) and you
add an extra load for ever ivar access.
That is an abomination.
Another stupid question : the memory allocation function accepts an
'extraBytes' parameter. These bytes are allocated _after_ the instance
bytes by the same 'malloc' call. Couldn't this system be adapted to ivar
storage ? The extra storage is at self+1 : no need for an additional
pointer, and no extra load for ivar access.
@interface NSObject (ExtraMemory)
+ (id) allocWithZone: (NSZone*)zone extra: (size_t) extraBytes;
@end
Usage
typedef
struct _LibraryClass_ext_st
{
int ivar1;
int ivar2;
} _LibraryClass_ext;
#define LIBRARYCLASS_EXTRA(obj) ((_LibraryClass_ext*)((obj)+1)))
@implementation LibraryClass
- (id) allocWithZone: (NSZone*)z
{
return [self allocWithZone: z extra: sizeof(_LibraryClass_ext)];
}
- (int) var1
{
return LIBRARYCLASS_EXTRA(self)-ivar1;
}
@end
___
Gnustep-dev mailing list
Gnustep-dev@gnu.org
http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 10 Mar 2009, at 10:28, Xavier Glattard wrote: Riccardo Mottola a écrit : (...) David Chisnall wrote: (...) Yes, this appears to me the least hurting path. But I laready dislike. If we really need, this should be the road. If you make private ivars into a structure and make a pointer to this an ivar, you add an extra malloc for every +alloc (expensive) and you add an extra load for ever ivar access. That is an abomination. Another stupid question : the memory allocation function accepts an 'extraBytes' parameter. These bytes are allocated _after_ the instance bytes by the same 'malloc' call. Couldn't this system be adapted to ivar storage ? The extra storage is at self+1 : no need for an additional pointer, and no extra load for ivar access. The extra bytes are allocated after the object, which would be completely useless. Imagine: Class A has 1 int ivar a.. Class B subclasses A and adds another int ivar b. The layout of B will be: id isa; // offset 0 int a; // offset 4 int b; // offset 8 Now you use the extrabytes feature to add space for another int, so you have: id isa; // offset 0 int a; // offset 4 int b; // offset 8 int extra; // offset 12 Now you add another ivar to A and remove the extra bytes, and you have this layout: id isa; // offset 0 int a; // offset 4 int a1; // offset 8 int b; // offset 12 - This has now moved and class B needs recompiling, as do any subclasses of B. Alternatively, you could try storing the data in the extra bytes, so the layout would be: id isa; // offset 0 int a; // offset 4 int b; // offset 8 int a1; // offset 12 That sounds sensible, unless you remember that there may be other subclasses of A. Imagine subclass C declares two doubles as ivars. Now you have three classes: A: id isa; // offset 0 int a; // offset 4 int a1; // offset 8 B: id isa; // offset 0 int a; // offset 4 int b; // offset 8 int a1; // offset 12 C: id isa; // offset 0 int a; // offset 4 double c1; // offset 8 double c2; // offset 16 int a1; // offset 24 We now have three different classes, with offsets of 8, 12 and 24, respectively for the a1 ivar. Every method that attempted to access this ivar would need to determine which class it is and calculate the offset. This would need a macro like this: #define a1 (*((int*)(((char*)self) +isa-instance_size))) This would be really horrible and error-prone (this macro doesn't take into account alignment, so is only valid for architectures like x86, with no strict alignment requirements (as long as none of the ivars are vectors), and would break on SPARC and similar archs. The next ivar you add would need an even more complex macro to account for the alignment of a1. This ivar would not be exposed via any introspection, so you couldn't use it with KVC/KVO, bindings, or EtoileUI without writing even more code. In short, this adds a lot more problems than it solves. The extra storage is not at self+1 unless self is cast to the correct subclass first. Pointer arithmetic like that only works when the size of the pointee is known at compile time, which is not the case for Objective- C objects, except in the trivial case of classes with no subclasses, which do not encounter this problem to start with. David ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
David Chisnall a écrit :
On 10 Mar 2009, at 10:28, Xavier Glattard wrote:
Riccardo Mottola a écrit :
(...)
David Chisnall wrote:
(...)
Yes, this appears to me the least hurting path. But I laready dislike.
If we really need, this should be the road.
If you make private ivars into a structure and make a pointer to this
an ivar, you add an extra malloc for every +alloc (expensive) and you
add an extra load for ever ivar access.
That is an abomination.
Another stupid question : the memory allocation function accepts an
'extraBytes' parameter. These bytes are allocated _after_ the instance
bytes by the same 'malloc' call. Couldn't this system be adapted to
ivar storage ? The extra storage is at self+1 : no need for an
additional pointer, and no extra load for ivar access.
The extra bytes are allocated after the object, which would be
completely useless. Imagine:
(...)
Now you add another ivar to A and remove the extra bytes, and you have
this layout:
id isa;// offset 0
int a;// offset 4
int a1;// offset 8
int b;// offset 12 - This has now moved and class B needs
recompiling, as do any subclasses of B.
Yes, but this is the case we want to avoid with the use of extra bytes!
-- do not insert ivar, add it in extra bytes
Alternatively, you could try storing the data in the extra bytes, so the
layout would be:
(...)
We now have three different classes, with offsets of 8, 12 and 24,
respectively for the a1 ivar. Every method that attempted to access
this ivar would need to determine which class it is and calculate the
offset. This would need a macro like this:
#define a1 (*((int*)(((char*)self) +isa-instance_size)))
This would be really horrible and error-prone (this macro doesn't take
into account alignment, so is only valid for architectures like x86,
with no strict alignment requirements (as long as none of the ivars are
vectors), and would break on SPARC and similar archs. The next ivar you
add would need an even more complex macro to account for the alignment
of a1. This ivar would not be exposed via any introspection, so you
couldn't use it with KVC/KVO, bindings, or EtoileUI without writing even
more code.
In short, this adds a lot more problems than it solves. The extra
storage is not at self+1 unless self is cast to the correct subclass
first. Pointer arithmetic like that only works when the size of the
pointee is known at compile time, which is not the case for Objective-C
objects, except in the trivial case of classes with no subclasses, which
do not encounter this problem to start with.
David
The extra bytes are already used in some class in GNUstep (GSString,
GSArray...) and AKAIK there is no problem with system arch.
In GSArray.m:417 :
_content_array = (id*)self[1];
So my 'self+1' was not so naive ;)
I have no idea of the macro that would be written for this task. Yours
is very ugly ;) But it'd have to be writen only once if the extra bytes
are defined as a structure. And then it would not be so error prone.
// In NSObject
#define OBJECT_EXTRA_PTR ((void*)(((char*)self) +isa-instance_size))
// In FooClass
struct _foo_extra_st
{
int a1;
} _foo_extra;
#define FOO_EXTRA (*(_foo_extra*)(OBJECT_EXTRA_PTR))
- (int) a1
{
return FOO_EXTRA.a1;
}
You may find two other examples in GSString.m: 633, 3087
Anyway that's only an idea, not a solution.
I'm convinced that the solution would be more 'political' than technical.
Thank you for your patience.
- Xavier
___
Gnustep-dev mailing list
Gnustep-dev@gnu.org
http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 10 Mar 2009, at 14:27, Xavier Glattard wrote:
David Chisnall a écrit :
The extra bytes are allocated after the object, which would be
completely useless. Imagine:
(...)
Now you add another ivar to A and remove the extra bytes, and you
have this layout:
id isa;// offset 0
int a;// offset 4
int a1;// offset 8
int b;// offset 12 - This has now moved and class B needs
recompiling, as do any subclasses of B.
Yes, but this is the case we want to avoid with the use of extra
bytes!
-- do not insert ivar, add it in extra bytes
Right.
Alternatively, you could try storing the data in the extra bytes,
so the layout would be:
(...)
We now have three different classes, with offsets of 8, 12 and 24,
respectively for the a1 ivar. Every method that attempted to
access this ivar would need to determine which class it is and
calculate the offset. This would need a macro like this:
#define a1 (*((int*)(((char*)self) +isa-instance_size)))
This would be really horrible and error-prone (this macro doesn't
take into account alignment, so is only valid for architectures
like x86, with no strict alignment requirements (as long as none of
the ivars are vectors), and would break on SPARC and similar
archs. The next ivar you add would need an even more complex macro
to account for the alignment of a1. This ivar would not be exposed
via any introspection, so you couldn't use it with KVC/KVO,
bindings, or EtoileUI without writing even more code.
In short, this adds a lot more problems than it solves. The extra
storage is not at self+1 unless self is cast to the correct
subclass first. Pointer arithmetic like that only works when the
size of the pointee is known at compile time, which is not the case
for Objective-C objects, except in the trivial case of classes with
no subclasses, which do not encounter this problem to start with.
David
The extra bytes are already used in some class in GNUstep (GSString,
GSArray...) and AKAIK there is no problem with system arch.
GSArray is a private GNUstep class. It is not exposed in headers
anywhere and can not be subclassed by anything external to GNUstep.
In GSArray.m:417 :
_content_array = (id*)self[1];
So my 'self+1' was not so naive ;)
See above. This only works in the special case where the class has no
subclasses. Create a subclass of GSArray, add an instance variable,
and watch everything go badly wrong.
This discussion is only relevant to classes which will be subclassed
in third-party apps and frameworks. Private classes are not part of
the public ABI and so are completely irrelevant. You can do whatever
you want with a class that won't be subclassed, or for which you
control all of the subclasses. You can't do any of these tricks with
a class intended for subclassing, and these classes are the topic of
the discussion.
I have no idea of the macro that would be written for this task.
Yours is very ugly ;) But it'd have to be writen only once if the
extra bytes are defined as a structure. And then it would not be so
error prone.
Then the structure becomes part of the ABI and we're back where we
started, only without introspection and reimplementing parts of the
language in an inefficient and error-prone way.
// In NSObject
#define OBJECT_EXTRA_PTR ((void*)(((char*)self) +isa-instance_size))
// In FooClass
struct _foo_extra_st
{
int a1;
} _foo_extra;
#define FOO_EXTRA (*(_foo_extra*)(OBJECT_EXTRA_PTR))
- (int) a1
{
return FOO_EXTRA.a1;
}
Right, and now try having two classes using this mechanism. The
subclass will need to know the offset from the parent.
It quickly becomes horribly unmaintainable, it adds an extra layer of
indirection, and provides no real benefits. It also breaks any
classes that use the extra bytes mechanism themselves. And, as I
said, it also breaks introspection, which will make all sorts of
things that use the more advanced features of Objective-C fail
silently and in a manner that is almost impossible to debug.
You may find two other examples in GSString.m: 633, 3087
GSString is another private class. It is completely irrelevant.
Anyway that's only an idea, not a solution.
Correct.
I'm convinced that the solution would be more 'political' than
technical.
Totally agree. In many cases, avoiding changing the ABI can be
avoided or postponed, but there needs to be an attitude that it is
something that we want to avoid, rather than something to do because
it's easy. Until we have non-fragile ivars in the runtime, anyway, at
which point the whole discussion becomes academic.
David
___
Gnustep-dev mailing list
Gnustep-dev@gnu.org
http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
David Chisnall a écrit :
On 10 Mar 2009, at 14:27, Xavier Glattard wrote:
David Chisnall a écrit :
The extra bytes are allocated after the object, which would be
completely useless. Imagine:
(...)
Now you add another ivar to A and remove the extra bytes, and you
have this layout:
id isa;// offset 0
int a;// offset 4
int a1;// offset 8
int b;// offset 12 - This has now moved and class B needs
recompiling, as do any subclasses of B.
Yes, but this is the case we want to avoid with the use of extra bytes!
-- do not insert ivar, add it in extra bytes
Right.
Alternatively, you could try storing the data in the extra bytes, so
the layout would be:
(...)
We now have three different classes, with offsets of 8, 12 and 24,
respectively for the a1 ivar. Every method that attempted to access
this ivar would need to determine which class it is and calculate the
offset. This would need a macro like this:
#define a1 (*((int*)(((char*)self) +isa-instance_size)))
This would be really horrible and error-prone (this macro doesn't
take into account alignment, so is only valid for architectures like
x86, with no strict alignment requirements (as long as none of the
ivars are vectors), and would break on SPARC and similar archs. The
next ivar you add would need an even more complex macro to account
for the alignment of a1. This ivar would not be exposed via any
introspection, so you couldn't use it with KVC/KVO, bindings, or
EtoileUI without writing even more code.
In short, this adds a lot more problems than it solves. The extra
storage is not at self+1 unless self is cast to the correct subclass
first. Pointer arithmetic like that only works when the size of the
pointee is known at compile time, which is not the case for
Objective-C objects, except in the trivial case of classes with no
subclasses, which do not encounter this problem to start with.
David
The extra bytes are already used in some class in GNUstep (GSString,
GSArray...) and AKAIK there is no problem with system arch.
GSArray is a private GNUstep class. It is not exposed in headers
anywhere and can not be subclassed by anything external to GNUstep.
In GSArray.m:417 :
_content_array = (id*)self[1];
So my 'self+1' was not so naive ;)
See above. This only works in the special case where the class has no
subclasses. Create a subclass of GSArray, add an instance variable, and
watch everything go badly wrong.
This discussion is only relevant to classes which will be subclassed in
third-party apps and frameworks. Private classes are not part of the
public ABI and so are completely irrelevant. You can do whatever you
want with a class that won't be subclassed, or for which you control all
of the subclasses. You can't do any of these tricks with a class
intended for subclassing, and these classes are the topic of the
discussion.
I have no idea of the macro that would be written for this task. Yours
is very ugly ;) But it'd have to be writen only once if the extra
bytes are defined as a structure. And then it would not be so error
prone.
Then the structure becomes part of the ABI and we're back where we
started, only without introspection and reimplementing parts of the
language in an inefficient and error-prone way.
// In NSObject
#define OBJECT_EXTRA_PTR ((void*)(((char*)self) +isa-instance_size))
// In FooClass
struct _foo_extra_st
{
int a1;
} _foo_extra;
#define FOO_EXTRA (*(_foo_extra*)(OBJECT_EXTRA_PTR))
- (int) a1
{
return FOO_EXTRA.a1;
}
Right, and now try having two classes using this mechanism. The
subclass will need to know the offset from the parent.
It quickly becomes horribly unmaintainable, it adds an extra layer of
indirection, and provides no real benefits. It also breaks any classes
that use the extra bytes mechanism themselves. And, as I said, it also
breaks introspection, which will make all sorts of things that use the
more advanced features of Objective-C fail silently and in a manner that
is almost impossible to debug.
You may find two other examples in GSString.m: 633, 3087
GSString is another private class. It is completely irrelevant.
Anyway that's only an idea, not a solution.
Correct.
I'm convinced that the solution would be more 'political' than technical.
Totally agree. In many cases, avoiding changing the ABI can be avoided
or postponed, but there needs to be an attitude that it is something
that we want to avoid, rather than something to do because it's easy.
Until we have non-fragile ivars in the runtime, anyway, at which point
the whole discussion becomes academic.
David
I'm sorry, i still can't see any problem.
NSAllocateObject is implemented as this : (NSObject.m:767)
size = aClass-instance_size + extraBytes + sizeof(struct obj_layout);
new = NSZoneMalloc(zone, size);
Then the extra bytes are allocated always _after_ the class ivar,
whatever the class is : the parent class or
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Hi, Gregory Casamento wrote: The last collective release was only two months ago. As far as the ABI is concerned that is certainly an issue. The last time we discussed it we came up with two solutions: snip I, personally, think we should implement the first option. It's the method most APIs follow and it is the method that is the most predictable. It would take some effort to do this, but it's minimal since it's really just padding the structures with a given amount of space. To the two solutions mentioned, there is David Chisnall's non-fragile ivars strategy. Now let me put down my points: - I do not want any additional runtime overhead. Performance needs to be maximum. Always. - I do not want to relay on some magic compiler and runtime trickery. I want to be easily compatible with the widest range from compilers, not only gcc 2.95, but also for example apple's compilers and who knows what else Why the two above? Portability, for example. Performance on embedded systems. (Nikolaus?) These are strengths we have and should further extend, not hamper them. So I would go for the passing solution and for clear releases. I essentially think it is not such a big problem if every change is clearly documented and minor and major releases are clear. As the library stabilizes, we break things less. The end user should just see a massive update in his package manager. This happens for gtk too... Riccardo ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 7 Mar 2009, at 08:30, Riccardo Mottola wrote: - I do not want any additional runtime overhead. Performance needs to be maximum. Always. Don't use Objective-C then. The language compromises performance for flexibility all of the time. How much in GNUstep is CPU-bound? Most of the bottlenecks I see are network and I/O-related. No overhead is not possible. All of the proposed solutions add overhead: If you add extra, unused, space in objects, you bloat memory usage and (much worse) data cache usage and trigger more cache misses and more paging on memory-constrained systems. If you make private ivars into a structure and make a pointer to this an ivar, you add an extra malloc for every +alloc (expensive) and you add an extra load for ever ivar access. Non-fragile ivars work by storing the offset in a global and setting it at runtime. This adds one word per ivar of memory usage (per class, not per instance), which is negligible. It also adds a load for every ivar access. This load is in the same location for every instance, so is likely to be in cache already for frequently-used classes / ivars. On x86 and ARM there are addressing modes that make this a very quick operation. - I do not want to relay on some magic compiler and runtime trickery. I want to be easily compatible with the widest range from compilers, not only gcc 2.95, but also for example apple's compilers and who knows what else Leopard shipped with two runtime, the `legacy' (NeXT) runtime and the `modern' (64-bit) one. The modern runtime supports non-fragile ivars, as do both of the compilers that Apple currently ships. Support for the modern runtime went into clang a few months back, so there are now four compilers (gcc, llvm-gcc, clang) capable of supporting non- fragile ivars on OS X. As for other compilers, GNUstep never supported POC and probably will never do so. GCC 2.95 is an abomination that needs to die. It is so far away from being standards-compliant in any modern sense that it's not even funny. If there are any platforms that actually work with GNUstep and have such an archaic compiler, then I would suggest that the best long-term strategy for supporting them is to use clang with LLVM's C back-end, and compile the result with the platform's C compiler. Keeping support for old platforms should not come at the expense of making GNUstep a first-rate platform for developing today. David ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Richard Frith-Macdonald a écrit : On 6 Mar 2009, at 07:05, Xavier Glattard wrote: Hi I dont understand this problem about ABI and binary compatibility, so please excuse me if this looks stupid :\ The problem comes from ivars that would not be always at the same offset, doesn't it ? But these ivars are supposed to be private, dont they ? So the code that uses these ivars is always released along with the class interface. I can not find any official 'public' ivars. Do i miss something ? There are all sorts of possible issues with symbol names when functions and global variables etc are added/removed/changed, but the main issue we are talking about is the ivar layout and subclassing. If you add an ivar to a class, the memory used by and instance becomes bigger to make room for it. If you link in some bundle or library which uses a subclass of that class, the bundle/library will not be expecting that ivar to be there, and will have stored one of the subclass ivars at the oiffset in memory where the new ivar is stored ... causing obvious problems. David's solution of non-fragile ivars requires compiler and runtime support, but is definitely the way to go to solve the problem (except for when the very highest performance is needed, in which case you can usually use straight C and other techniques for optimisation). Fred, Richard : thank you for your answers. I think I understand the 'technical' problem :) But IMHO : - private ivars should (can?) not be used from outside the class itself. - protected ivars should only be used from inside the library/framework itself, and only for very good reason. - public ivars should not exist! Moreover ivars are never documented. Then GNUstep users should never meet this problem. If they do then I think this is their own choice. Do i still miss something ? -- Regards ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Richard Frith-Macdonald a écrit :
On 6 Mar 2009, at 09:15, Xavier Glattard wrote:
Richard Frith-Macdonald a écrit :
On 6 Mar 2009, at 07:05, Xavier Glattard wrote:
Hi
I dont understand this problem about ABI and binary compatibility, so
please excuse me if this looks stupid :\
The problem comes from ivars that would not be always at the same
offset, doesn't it ? But these ivars are supposed to be private, dont
they ? So the code that uses these ivars is always released along with
the class interface. I can not find any official 'public' ivars. Do i
miss something ?
There are all sorts of possible issues with symbol names when
functions and global variables etc are added/removed/changed, but the
main issue we are talking about is the ivar layout and subclassing.
If you add an ivar to a class, the memory used by and instance
becomes bigger to make room for it.
If you link in some bundle or library which uses a subclass of that
class, the bundle/library will not be expecting that ivar to be
there, and will have stored one of the subclass ivars at the oiffset
in memory where the new ivar is stored ... causing obvious problems.
David's solution of non-fragile ivars requires compiler and runtime
support, but is definitely the way to go to solve the problem (except
for when the very highest performance is needed, in which case you
can usually use straight C and other techniques for optimisation).
Fred, Richard : thank you for your answers.
I think I understand the 'technical' problem :)
But IMHO :
- private ivars should (can?) not be used from outside the class itself.
- protected ivars should only be used from inside the
library/framework itself, and only for very good reason.
- public ivars should not exist!
Moreover ivars are never documented.
Then GNUstep users should never meet this problem. If they do then I
think this is their own choice.
Do i still miss something ?
(...)
but the actual layout at runtime is
{
Class isa;
int _ivar1;
int _ivar2;
NSString *name;
}
so the assignment actually overwrites _ivar2
(...)
I have think of this, but i could not consider the compiler is so stupid!
Thank you for your patience.
Anyway, do you think we should rely on a (non-existent) compiler feature
? It takes a long time for new compiler release to reach end users. Is
the compatibility with gcc 2.95 a forgotten dream ? ;)
Do you think a backward compatible version of some classes might be
created at each release ? Additional ivars would be stored externally in
a map table : slow but simple. This version of the class would be loaded
at runtime, but would be private and only the new version could be used
by a compiler. The implementation file could be easily shared by both
version (if not the implementation itself with the use of accessors)
==
LibraryClass.h
#ifndef __GSLibraryClass_Version_flag__
#define __GSLibraryClass_Version_flag__
int __GSLibraryClass_mandatory_version = 0x0200; // not extern !
#enfif
@class LibraryClass_old;
@interface LibraryClass: LibraryClass_old
{
@private
// int_ivar1; -- inherited
int_ivar2;
}
- int var2;
- int twice;
@end
LibraryClass.m
#define __GSLibraryClass_Version_flag__
extern int __GSLibraryClass_mandatory_version;
static int __GSLibraryClass_version = 0x0200;
#include LibraryClass.h
@interface LibraryClass_old : NSObject
{
@protected
int_ivar1;
}
@end
@implementation LibraryClass_old
static GSIMap * _ivar2_store;
- int var2
{ return [_ivar2_store valueForKey: self]; }
- int twice
{
return [self var2] * 2;
}
@end
@implementation LibraryClass
+ (id) alloc
{
if ( __GSLibraryClass_version != __GSLibraryClass_mandatoryVersion )
{
return [LibraryClass_old alloc];
}
else
{
return [super alloc];
}
}
- int var2
{ return _ivar2; }
// if needed :
- int twice
{
return _ivar2 * 2;
}
@end
=
Some good old MACROS might be helpful. I know there are some MACROS
gurus around ;)
Just an idea. Maybe too complicated.
-- Regards
- Xavier
___
Gnustep-dev mailing list
Gnustep-dev@gnu.org
http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Am Freitag, den 06.03.2009, 06:17 + schrieb Richard Frith-Macdonald: I feel we probably need to break things in trunk during the development cycle to get a reaction and some suggestions from other people. For instance I asked for ideas about the change to using NSUInteger,NSInteger, and CGFloat, and I adopted your idea of a #define to retain the old style behacvior, but since then I've seen no feedback about making this change work with gui/back and other apps. What I probably need to do is switch that code to use the new Apple behavior by default, deliberately breaking 64bit systems so that people will do something about it, or will at least sned specific bug reports for me to deal with. FWIW, I think I'm fine with changing the default, yet hope the other option remains. But since you said you received no feedback I want to reiterate another suggestion: Instead of: #if defined(GS_64BIT_OLD) typedef int NSInteger; typedef unsigned intNSUInteger; typedef float CGFloat; which produces a lot of compiler warning for code targeting both GNUstep and legacy API, would you consider: #define NSInteger int; #define NSUInteger unsigned int; #define CGFloat float; the bonus: it stops the warnings: Code targeting older API's will compile without the noise. the drawback: it stops the warnings: Code being my not be upgraded to use the new types so that future versions will be compatible with 64 bit. Personally I can understand if you believe the #define may be worse and I can surely keep that patch locally. But then again, anyone wanting to stay up to date will hardly use the define/configure option. Cheers, David ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Hi I dont understand this problem about ABI and binary compatibility, so please excuse me if this looks stupid :\ The problem comes from ivars that would not be always at the same offset, doesn't it ? But these ivars are supposed to be private, dont they ? So the code that uses these ivars is always released along with the class interface. I can not find any official 'public' ivars. Do i miss something ? And if the ivars need to be used from outside the library/bundle (for internal use), why not access them thru accessors? Obj-C accessors or even simple C function that would read or write the value, or return a pointer, or even a member pointer (offset)... My 2 cts. -- Regards - Xavier Gregory Casamento a écrit : The last collective release was only two months ago. As far as the ABI is concerned that is certainly an issue. The last time we discussed it we came up with two solutions: * Pad the ivar-structures in the classes out to give space to grow so that it pushes off any ABI compatibility issues as long as possible. This is why in some APIs, including Cocoa, you see things like reserved... or private... variables. These are there to give room to grow. The disadvantage is that the classes would then take up more memory as a result. * Move the ivar-structures out of the classes and replace them with a void pointer to the actual structure. This has the advantage that we will never be able to break ABI compatibilty since the sizes of the structs in the classes will not change... but it also has the disadvantage of adding a layer of complexity to getting and setting variables as well as potentially causing unpredictable issues due to unforseen incompatibilities such as cases where the wrong data is written into a data structure causing some sort of corruption when using the wrong version of a library. I, personally, think we should implement the first option. It's the method most APIs follow and it is the method that is the most predictable. It would take some effort to do this, but it's minimal since it's really just padding the structures with a given amount of space. GC P.S. Sorry for the double mail, Stefan, I meant to reply to all, but curse gmail it defaults to reply. :) On Thu, Mar 5, 2009 at 8:35 AM, Stefan Bidigaray stefanb...@gmail.com mailto:stefanb...@gmail.com wrote: I thought that's what GNUstep-startup was supposed to be used for? I've personally never used it, but the way I understand it all you need to do is ./InstallGNUstep, right? In my opinion, it's more important to have up-to-date packages than anything else, however, one needs to keep in mind that most distributions do not follow the latest and greatest. I, for example, use Debian Testing + GNOME most of the time and I know I will never have the latest release of packages, people need to realize this. Tell you the truth, except for when I create GNUstep packages and build scripts for Slackware, I never compile a single piece of software. Why is it different with GNUstep? Why do people WANT to compile in the first place? If I can rant a little... GNUstep also gets way behind when it comes to releases. How long has it been since the last collective release? Why isn't GNUstep releasing more stable releases? It seems everytime a new release of the core packages are out I need to recompile everything because it breaks ABI. Rarely do I see a x.x.1 release. I've been wanting to bring that up for a while, figured this was as a good a time as any. Stefan On Wed, Mar 4, 2009 at 10:33 PM, Gregory Casamento greg.casame...@gmail.com mailto:greg.casame...@gmail.com wrote: Hey guys... it seems to me that the build guides for building from source that we have are out of date. Could we move some of this stuff to the wiki so that it can be more readily maintained? Also, it seems like a lot of users are having issues using GNUstep because the packages are so old on many systems. Currently Hubert is working on What brought this up is that I was trying to help someone get gnustep installed since the packages are out of date. Anyone have any thoughts? GC -- Gregory Casamento Open Logic Corporation, Principal Consultant ## GNUstep Chief Maintainer yahoo/skype: greg_casamento, aol: gjcasa (240)274-9630 (Cell), (301)362-9640 (Home) ___ Gnustep-dev mailing list Gnustep-dev@gnu.org mailto:Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev -- Gregory Casamento Open Logic Corporation, Principal Consultant ## GNUstep Chief Maintainer yahoo/skype: greg_casamento, aol:
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 5 Mar 2009, at 16:10, Gregory Casamento wrote: The last collective release was only two months ago. As far as the ABI is concerned that is certainly an issue. The last time we discussed it we came up with two solutions: • Pad the ivar-structures in the classes out to give space to grow so that it pushes off any ABI compatibility issues as long as possible. This is why in some APIs, including Cocoa, you see things like reserved... or private... variables. These are there to give room to grow. The disadvantage is that the classes would then take up more memory as a result. And oftentimes the padding is insufficient for what you want to do. So this mechanism alone can't make the code future-proof. • Move the ivar-structures out of the classes and replace them with a void pointer to the actual structure. This has the advantage that we will never be able to break ABI compatibilty since the sizes of the structs in the classes will not change... but it also has the disadvantage of adding a layer of complexity to getting and setting variables True. The code complexity is easily overcome by using the C preprocessor, but it's a real pain trying to examine such classes from within gdb. as well as potentially causing unpredictable issues due to unforseen incompatibilities such as cases where the wrong data is written into a data structure causing some sort of corruption when using the wrong version of a library. That can't happen ... if you are using a different version of the library then it's using its own different version of the data structure. The big disadvantage of this approach is (in some cases) performance. You have to do twice as much memory allocation/deallocation sincethe object and the structure holding its ivars are separate. This is really bad news for objects which are created/destroyed frequently, but irrelevant for other objects with long lifetimes. I, personally, think we should implement the first option. It's the method most APIs follow and it is the method that is the most predictable. It would take some effort to do this, but it's minimal since it's really just padding the structures with a given amount of space. You missed one of the most common strategies used in Cocoa ... declaring the public API as an abstract base class with no instance variables, and returning instances of privates subclasses. I dislike that as it makes it impossible to properly subclass many classes, but as we are aiming for Apple compatibility it's not really a disadvantege (people can't subclass easily in Cocoa either). Where apple use class clusters, we may as well do the same thing. Otherwise, where we are adding new classes we should probably use a pointer to private data for long lived and complex classes with a lot of instance variables, but use padding (reserve a pointer too as part of the padding) for simpler classes with a short lifetime. With the void* pointer to the structure we might be able to do something clever to help debugging. I haven't tried it, but we could define a preprocessor symbol (eg. GSRESERVED) which normally expands to 'void*' (in the public API) but expands to 'struct foo*' inside the source files, so that the symbol tables in the compiled code know what the internal variable layout actually is, and gdb can therefore make sense of things when examining the an object. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 5 Mar 2009, at 16:10, Gregory Casamento wrote: The last collective release was only two months ago. As far as the ABI is concerned that is certainly an issue. The last time we discussed it we came up with two solutions: • Pad the ivar-structures in the classes out to give space to grow so that it pushes off any ABI compatibility issues as long as possible. This is why in some APIs, including Cocoa, you see things like reserved... or private... variables. These are there to give room to grow. The disadvantage is that the classes would then take up more memory as a result. • Move the ivar-structures out of the classes and replace them with a void pointer to the actual structure. This has the advantage that we will never be able to break ABI compatibilty since the sizes of the structs in the classes will not change... but it also has the disadvantage of adding a layer of complexity to getting and setting variables as well as potentially causing unpredictable issues due to unforseen incompatibilities such as cases where the wrong data is written into a data structure causing some sort of corruption when using the wrong version of a library. Or, option 3, use non-fragile ivars. I plan on adding support for this to the GNU runtime and the clang implementation over the next few weeks. It can be done without breaking the existing GNU runtime ABI an, although it won't work retroactively, will let us change the ivar layout of classes without subclasses requiring recompilation. (Oh, please, please, don't do option 2 - it would break a lot of the introspection that we use heavily in Étoilé). Of course, this only applies to classes. When people decide to remove global variables that are referenced by static functions declared in headers, this will still break the ABI, and really shouldn't have been committed without a period of at least a year while the relevant symbol was marked as deprecated. David ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
On 5 Mar 2009, at 20:27, David Chisnall wrote: On 5 Mar 2009, at 16:10, Gregory Casamento wrote: The last collective release was only two months ago. As far as the ABI is concerned that is certainly an issue. The last time we discussed it we came up with two solutions: • Pad the ivar-structures in the classes out to give space to grow so that it pushes off any ABI compatibility issues as long as possible. This is why in some APIs, including Cocoa, you see things like reserved... or private... variables. These are there to give room to grow. The disadvantage is that the classes would then take up more memory as a result. • Move the ivar-structures out of the classes and replace them with a void pointer to the actual structure. This has the advantage that we will never be able to break ABI compatibilty since the sizes of the structs in the classes will not change... but it also has the disadvantage of adding a layer of complexity to getting and setting variables as well as potentially causing unpredictable issues due to unforseen incompatibilities such as cases where the wrong data is written into a data structure causing some sort of corruption when using the wrong version of a library. Or, option 3, use non-fragile ivars. I plan on adding support for this to the GNU runtime and the clang implementation over the next few weeks. It can be done without breaking the existing GNU runtime ABI an, although it won't work retroactively, will let us change the ivar layout of classes without subclasses requiring recompilation. (Oh, please, please, don't do option 2 - it would break a lot of the introspection that we use heavily in Étoilé). That's interesting. What sort of introspection do you do which needs to know about the layout of private ivars? Shouldn't the KVC/KVO facilities be enough? Of course, this only applies to classes. When people decide to remove global variables that are referenced by static functions declared in headers, this will still break the ABI, and really shouldn't have been committed without a period of at least a year while the relevant symbol was marked as deprecated. You are referring to the NSZone thing? Of course that hasn't been done. We need people checking/using svn trunck to make sure we deal with any temporary breakage (or any completely accidental breakage) before a release. I feel we probably need to break things in trunk during the development cycle to get a reaction and some suggestions from other people. For instance I asked for ideas about the change to using NSUInteger,NSInteger, and CGFloat, and I adopted your idea of a #define to retain the old style behacvior, but since then I've seen no feedback about making this change work with gui/back and other apps. What I probably need to do is switch that code to use the new Apple behavior by default, deliberately breaking 64bit systems so that people will do something about it, or will at least sned specific bug reports for me to deal with. ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
Re: ABI Compatibility (was Re: Installation woes for the average user...)
Xavier Glattard wrote: The problem comes from ivars that would not be always at the same offset, doesn't it ? But these ivars are supposed to be private, dont they ? So the code that uses these ivars is always released along with the class interface. I can not find any official 'public' ivars. Do i miss something ? And if the ivars need to be used from outside the library/bundle (for internal use), why not access them thru accessors? Obj-C accessors (KVC) or even simplefast C functions that would read or write the value, or return a pointer, or even a member pointer (offset)... The problem is with subclasses, say you have a subclass of NSCell and GNUstep changes the ivar layout of NSCell by adding a new ivar. You will have to recompile your application with your subclass of NSCell to adopt to our change. What Greg, Richard and David are looking for are different solutions for this problem. Fred ___ Gnustep-dev mailing list Gnustep-dev@gnu.org http://lists.gnu.org/mailman/listinfo/gnustep-dev
