VOTE: Principles of sharing code
As suggested by many of you, here is a call vote on the overall intention for how we want to share code, without bogging down in the details. Thanks to Kathey for a very helpful suggested wording. This vote is to support the overall intention to implement a framework to allow code sharing across the Derby product jars that will allow Derby jars of the same major version to be loaded in a Java VM without having to use specialized classloaders. Forward and backward compatibility between jar files within the same major version will be implemented using versioning, compatibility tests, and an explicit procedure for deprecating internal APIs. If the major version of the jars loaded in a JVM differ, the user will need to separate the two versions by using separate classloaders. This implementation will not have any significant impact on jar file size or otherwise affect product distribution or usage. The detailed guidelines are being documented and refined on the Derby Wiki site at http://wiki.apache.org/db-derby/SharedComponentVersioningGuidelines. Subsequent to approval of this vote, a working implementation of shared code using this framework will be submitted for review. begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
+1
Re: VOTE: Principles of sharing code
+1 David W. Van Couvering wrote: As suggested by many of you, here is a call vote on the overall intention for how we want to share code, without bogging down in the details. Thanks to Kathey for a very helpful suggested wording. This vote is to support the overall intention to implement a framework to allow code sharing across the Derby product jars that will allow Derby jars of the same major version to be loaded in a Java VM without having to use specialized classloaders. Forward and backward compatibility between jar files within the same major version will be implemented using versioning, compatibility tests, and an explicit procedure for deprecating internal APIs. If the major version of the jars loaded in a JVM differ, the user will need to separate the two versions by using separate classloaders. This implementation will not have any significant impact on jar file size or otherwise affect product distribution or usage. The detailed guidelines are being documented and refined on the Derby Wiki site at http://wiki.apache.org/db-derby/SharedComponentVersioningGuidelines. Subsequent to approval of this vote, a working implementation of shared code using this framework will be submitted for review.
Re: VOTE: Principles of sharing code
+1
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > As suggested by many of you, here is a call vote on the overall > intention for how we want to share code, without bogging down in the > details. Thanks to Kathey for a very helpful suggested wording. > > This vote is to support the overall intention to implement a framework > to allow code sharing across the Derby product jars that will allow > Derby jars of the same major version to be loaded in a Java VM without > having to use specialized classloaders. Forward and backward > compatibility between jar files within the same major version will be > implemented using versioning, compatibility tests, and an explicit > procedure for deprecating internal APIs. > > If the major version of the jars loaded in a JVM differ, the user will > need to separate the two versions by using separate classloaders. Is this a requirement, 'will need', since the wiki page says "We should strive for forward compatibility between major releases, ..." and elsewhere in referring to incompatibility between major releases it says "(and this is to be avoided)"? > > This implementation will not have any significant impact on jar file > size or otherwise affect product distribution or usage. > > The detailed guidelines are being documented and refined on the Derby > Wiki site at > http://wiki.apache.org/db-derby/SharedComponentVersioningGuidelines. I'm a little nervous of the use of and reliance on "major version", since we don't have any guidelines as to when we would change major version number. One could read this vote as saying I can break the compatibility at any time by just bumping the major version. Is shared code really the driver for major version changes? Does specifying major version add any value to your proposal? Maybe you could take the approach that at all times we strive for compatibility. If someone in the future wants a change that would cause some compatibility issues, they could raise it then with a vote. Dan.
Re: VOTE: Principles of sharing code
Daniel John Debrunner wrote: David W. Van Couvering wrote: As suggested by many of you, here is a call vote on the overall intention for how we want to share code, without bogging down in the details. Thanks to Kathey for a very helpful suggested wording. This vote is to support the overall intention to implement a framework to allow code sharing across the Derby product jars that will allow Derby jars of the same major version to be loaded in a Java VM without having to use specialized classloaders. Forward and backward compatibility between jar files within the same major version will be implemented using versioning, compatibility tests, and an explicit procedure for deprecating internal APIs. If the major version of the jars loaded in a JVM differ, the user will need to separate the two versions by using separate classloaders. Is this a requirement, 'will need', since the wiki page says "We should strive for forward compatibility between major releases, ..." and elsewhere in referring to incompatibility between major releases it says "(and this is to be avoided)"? Perhaps I misunderstood, but I thought Kathey felt that if we did not *guarantee* compatibility then it would be simpler and more correct to say that a user should expect and plan for incompatibility between major releases, rather than hope and pray for compatibility. I would prefer to word it something like "may need" instead of "will need" but I was concerned about pushback from Kathey :) This implementation will not have any significant impact on jar file size or otherwise affect product distribution or usage. The detailed guidelines are being documented and refined on the Derby Wiki site at http://wiki.apache.org/db-derby/SharedComponentVersioningGuidelines. I'm a little nervous of the use of and reliance on "major version", since we don't have any guidelines as to when we would change major version number. One could read this vote as saying I can break the compatibility at any time by just bumping the major version. Is shared code really the driver for major version changes? Does specifying major version add any value to your proposal? Maybe you could take the approach that at all times we strive for compatibility. If someone in the future wants a change that would cause some compatibility issues, they could raise it then with a vote. Hm. I do know that it's standard practice to say that incompatibilities of any sort of interface be cause for a bump in major version. I personally would have interpreted this as saying that if someone attempts to introduce an incompatibility they will have to wait until the next major release, just as if someone were to propose changing the database file format in an incompatible way or modifying the 'ij' interface in some incompatible way. In other words, the intent was to provide pushback for incompatible changes, not motivation for more frequent major releases. What I could say is something like "at all time we strive for compatibility. If someone in the future wants a change that would cause some compatibility issues, they could raise it with a vote. Such a change would require a bump in the major release." The last sentence is consistent with the the Jakarta Commons versioning guidelines at http://jakarta.apache.org/commons/releases/versioning.html -- "Major releases signify significant changes to a component. Developers *may* perform a major release if there have been substantial improvements to the component. Developers *must* perform a major release whenever the new release is not at least interface-compatible the previous release." My point being that our guidelines need to allow customers to be assured that that they will *never* see interface incompatibilities between minor releases, whereas there is a *possibility* that there is an incompatibility between major releases. Dan. begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > > > Daniel John Debrunner wrote: > >> David W. Van Couvering wrote: >> >> Is this a requirement, 'will need', since the wiki page says "We should >> strive for forward compatibility between major releases, ..." and >> elsewhere in referring to incompatibility between major releases it says >> "(and this is to be avoided)"? >> > > Perhaps I misunderstood, but I thought Kathey felt that if we did not > *guarantee* compatibility then it would be simpler and more correct to > say that a user should expect and plan for incompatibility between major > releases, rather than hope and pray for compatibility. > > I would prefer to word it something like "may need" instead of "will > need" but I was concerned about pushback from Kathey :) Do you have a link to Kathey's e-mail, I looked in the archives (boy do I hate the new gui) but couldn't see what you are referring to? Thanks, Dan.
Re: VOTE: Principles of sharing code
Daniel John Debrunner <[EMAIL PROTECTED]> writes: > David W. Van Couvering wrote: >> >> >> Daniel John Debrunner wrote: >> >>> David W. Van Couvering wrote: > >>> >>> Is this a requirement, 'will need', since the wiki page says "We should >>> strive for forward compatibility between major releases, ..." and >>> elsewhere in referring to incompatibility between major releases it says >>> "(and this is to be avoided)"? >>> >> >> Perhaps I misunderstood, but I thought Kathey felt that if we did not >> *guarantee* compatibility then it would be simpler and more correct to >> say that a user should expect and plan for incompatibility between major >> releases, rather than hope and pray for compatibility. >> >> I would prefer to word it something like "may need" instead of "will >> need" but I was concerned about pushback from Kathey :) > > Do you have a link to Kathey's e-mail, I looked in the archives (boy do > I hate the new gui) but couldn't see what you are referring to? For discussions of this size only Google can help you... I think this was the article: http://www.nabble.com/Re%3A-VOTE%3A-Shared-Components-Guidelines-p1187186.html -- Knut Anders
Re: VOTE: Principles of sharing code
On 10/24/05, Daniel John Debrunner <[EMAIL PROTECTED]> wrote: Do you have a link to Kathey's e-mail, I looked in the archives (boy doI hate the new gui) but couldn't see what you are referring to?Thanks,Dan. Nabble is your friend ;)
Re: VOTE: Principles of sharing code
Hi, all. I wanted to let you know I plan to close the polls on this vote Friday midnight USA Pacific Time. Thanks, David David W. Van Couvering wrote: As suggested by many of you, here is a call vote on the overall intention for how we want to share code, without bogging down in the details. Thanks to Kathey for a very helpful suggested wording. This vote is to support the overall intention to implement a framework to allow code sharing across the Derby product jars that will allow Derby jars of the same major version to be loaded in a Java VM without having to use specialized classloaders. Forward and backward compatibility between jar files within the same major version will be implemented using versioning, compatibility tests, and an explicit procedure for deprecating internal APIs. If the major version of the jars loaded in a JVM differ, the user will need to separate the two versions by using separate classloaders. This implementation will not have any significant impact on jar file size or otherwise affect product distribution or usage. The detailed guidelines are being documented and refined on the Derby Wiki site at http://wiki.apache.org/db-derby/SharedComponentVersioningGuidelines. Subsequent to approval of this vote, a working implementation of shared code using this framework will be submitted for review. begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
Knut Anders Hatlen wrote: > Daniel John Debrunner <[EMAIL PROTECTED]> writes: > > >>David W. Van Couvering wrote: >> >>> >>>Daniel John Debrunner wrote: >>> >>> David W. Van Couvering wrote: >> Is this a requirement, 'will need', since the wiki page says "We should strive for forward compatibility between major releases, ..." and elsewhere in referring to incompatibility between major releases it says "(and this is to be avoided)"? >>> >>>Perhaps I misunderstood, but I thought Kathey felt that if we did not >>>*guarantee* compatibility then it would be simpler and more correct to >>>say that a user should expect and plan for incompatibility between major >>>releases, rather than hope and pray for compatibility. >>> >>>I would prefer to word it something like "may need" instead of "will >>>need" but I was concerned about pushback from Kathey :) >> >>Do you have a link to Kathey's e-mail, I looked in the archives (boy do >>I hate the new gui) but couldn't see what you are referring to? > > > For discussions of this size only Google can help you... > > I think this was the article: > > http://www.nabble.com/Re%3A-VOTE%3A-Shared-Components-Guidelines-p1187186.html Assuming that is what David was referring to, I see this quote: > Kathey said ... > That's just a stab at it, but I think it has most of the parts that are > really controversial, covers the significant risk areas, and makes it > clear that users can no longer mix jars across major versions. I think > it is good to be clear about that since as I see it, that is currently > the only proposed change to external behaviour. The Wiki page says that > developers should try to avoid breaking compatiblity across major > versions and that's fine, but makes no committment, so it is best if > we just document that it won't work. I disagree that we take the approach that since we can't guarantee compatibility then we document that it won't work. To follow that thought to the extreme, that would mean that if a bump in major version means some incompatible application api, then we should document that applications will not work between major versions. I much prefer the approach in the wiki page, strive for compatibility. If such a requirement comes up for a specific release, then we can document that sharing jars between 12.x and 13.x is not supported. Dan.
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > Hi, all. I wanted to let you know I plan to close the polls on this > vote Friday midnight USA Pacific Time. > I haven't actually seen any binding votes on this proposal yet. But may have missed them.
Re: VOTE: Principles of sharing code
Daniel John Debrunner wrote: > >I much prefer the approach in the wiki page, strive for compatibility. >If such a requirement comes up for a specific release, then we can >document that sharing jars between 12.x and 13.x is not supported. > > > Sounds ok to me as long as such an incompatibility is documented and in the absence of such documentation is treated as a bug.I'd just like the user impact to be clear in the documentation and not have a situation where mixing jars breaks and we say "Hey we really do strive for compatibility as much as possible, but we never guaranteed it would work." Don't the internal interface deprecation guidelines mean that mixed jars will stop working together at some point? BTW, if this thing gets submitted to vote again. I'd like to suggest we get consensus on the wording before it does. I'd much prefer David just submit the code to share messages with some really clear way to version the messages into the future that doesn't break anything and then you'd get no trouble from me. Kathey
Re: VOTE: Principles of sharing code
I am still learning, so forgive me if I missed something, but I carefully read the guidelines of our project: http://db.apache.org/decisions.html: "Changes to the products of the Project, including code and documentation, will appear as action items in the status file. All product changes to the currently active repository are subject to lazy consensus." and "An action requiring consensus approval must receive at least 3 binding +1 votes and no binding vetos. An action requiring majority approval must receive at least 3 binding +1 votes and more +1 votes than -1 votes. ==My emphasis== All other action items are considered to have lazy approval until somebody votes -1, after which point they are decided by either consensus or majority vote, depending on the type of action item.== end emphasis ==" Given that this is a change to the products of the Product, and nobody has voted -1, then I thought this meant lazy approval. Lazy approval is further described in the Apache Way: http://www.apache.org/foundation/how-it-works.html#management "When coordination is required, decisions are taken with a lazy consensus approach: a few positive votes with no negative vote is enough to get going." My understanding is that binding votes are only relevant when someone votes -1... Thanks, David Kathey Marsden wrote: David W. Van Couvering wrote: Hi, all. I wanted to let you know I plan to close the polls on this vote Friday midnight USA Pacific Time. I haven't actually seen any binding votes on this proposal yet. But may have missed them. begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > I am still learning, so forgive me if I missed something, but I > carefully read the guidelines of our project: > One could also claim that this paragraph applies to these principles: http://db.apache.org/source.html Doubtful changes, new features, and large scale overhauls need to be discussed before committing them into the repository. Any change that affects the semantics of an existing API function, the size of the program, configuration data formats, or other major areas must receive consensus approval before being committed. Which would then require three binding votes ... My issue with the currrent principles is that there was lots of good discussion which lead to the good, generally agreed upon, summary on the wiki. Then a single comment from a single individual leads to principles that don't match the summary in the wiki. I believe the principles should have been in-line with the wiki, since it was based upon discussion, then Kathey could have vetoed with a reason if required. The abrupt change really concerns me. Dan.
Re: VOTE: Principles of sharing code
Well, I am very confused, I could have sworn you had suggested this very approach of getting a vote on the principles of the thing, and then as a second phase submitting code that demonstrates an initial implementation. In terms of getting the wording right, I can see your point, but to be honest I thought I *had* the wording right. I'm not sure how I can know absolutely positively that I have everyone's buyin before submitting for a vote without actually having a vote. If you all would rather see code, fine, but as I understood it the suggestion I got from Dan (with a +1 from Andrew) was to do things incrementally, by voting on the overall principles first and then submitting a patch that can be reviewed/approved using the standard mechanisms. --- It seems to me we are very very close to having something people are happy with. If I were to change the wording FROM "If the major version of the jars loaded in a JVM differ, the user will need to separate the two versions by using separate classloaders. " TO "If the major version of the jars loaded in a JVM differ, the user may need to separate the two versions by using separate classloaders. If an incompatibility exists, it will be clearly documented. Any undocumented incompatibilities will be treated as a bug." would anyone be unhappy with this? PLEASE let me know by Friday. If I don't hear any objections I will put it up for (sigh) another vote. To answer your question about deprecation, yes, mixed jars between major versions may not work together at some point. But just because say jar files between 10.0 and 12.0 do not work together does not mean that jar files between 11.0 and 12.0 or between 11.0 and 13.0 will not work together. It does mean that jar files between 13.0 and 10.0 will not work together. David For your reference, here is the complete amended text below. == This vote is to support the overall intention to implement a framework to allow code sharing across the Derby product jars that will allow Derby jars of the same major version to be loaded in a Java VM without having to use specialized classloaders. Forward and backward compatibility between jar files within the same major version will be implemented using versioning, compatibility tests, and an explicit procedure for deprecating internal APIs. If the major version of the jars loaded in a JVM differ, the user may need to separate the two versions by using separate classloaders. If an incompatibility exists, it will be clearly documented. Any undocumented incompatibilities will be treated as a bug. This implementation will not have any significant impact on jar file size or otherwise affect product distribution or usage. The detailed guidelines are being documented and refined on the Derby Wiki site at http://wiki.apache.org/db-derby/SharedComponentVersioningGuidelines. Kathey Marsden wrote: Daniel John Debrunner wrote: I much prefer the approach in the wiki page, strive for compatibility. If such a requirement comes up for a specific release, then we can document that sharing jars between 12.x and 13.x is not supported. Sounds ok to me as long as such an incompatibility is documented and in the absence of such documentation is treated as a bug.I'd just like the user impact to be clear in the documentation and not have a situation where mixing jars breaks and we say "Hey we really do strive for compatibility as much as possible, but we never guaranteed it would work." Don't the internal interface deprecation guidelines mean that mixed jars will stop working together at some point? BTW, if this thing gets submitted to vote again. I'd like to suggest we get consensus on the wording before it does. I'd much prefer David just submit the code to share messages with some really clear way to version the messages into the future that doesn't break anything and then you'd get no trouble from me. Kathey begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
OK, thanks, I agree it does require consensus approval. I am glad my parents didn't try to make me a lawyer. I didn't see anyone complain about Kathey's comment so I thought it was an acceptable change. But it's not surprising that it wasn't: for some reason almost every intuition I have had about this process has been wrong. I'm not sure what you mean that it "concerns you" -- this seems to imply to me that you are worried about something in the future. Can you explain? Thanks, David Daniel John Debrunner wrote: David W. Van Couvering wrote: I am still learning, so forgive me if I missed something, but I carefully read the guidelines of our project: One could also claim that this paragraph applies to these principles: http://db.apache.org/source.html Doubtful changes, new features, and large scale overhauls need to be discussed before committing them into the repository. Any change that affects the semantics of an existing API function, the size of the program, configuration data formats, or other major areas must receive consensus approval before being committed. Which would then require three binding votes ... My issue with the currrent principles is that there was lots of good discussion which lead to the good, generally agreed upon, summary on the wiki. Then a single comment from a single individual leads to principles that don't match the summary in the wiki. I believe the principles should have been in-line with the wiki, since it was based upon discussion, then Kathey could have vetoed with a reason if required. The abrupt change really concerns me. Dan. begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > OK, thanks, I agree it does require consensus approval. I am glad my > parents didn't try to make me a lawyer. > > I didn't see anyone complain about Kathey's comment so I thought it was > an acceptable change. But it's not surprising that it wasn't: for some > reason almost every intuition I have had about this process has been wrong. It's interesting to look at why no-one discussed Kathey's comment, I certainly missed it. I think it's a case of an important comment being hidden by the rest of the e-mail. The majority of that e-mail http://www.nabble.com/Re%3A-VOTE%3A-Shared-Components-Guidelines-p1187186.html is about the process, along with an *example* vote text "Perhaps something like...". I certainly didn't read the following text because it was an example of how to word it, no point reading it (in my mind) as the actual vote text from you might be quite different. Then after the example text, and more text that is general guidance on approach, in the same paragraph is Kathey's objection. Too well hidden. > I'm not sure what you mean that it "concerns you" -- this seems to imply > to me that you are worried about something in the future. Can you explain? I said "The abrupt change really concerns me". I think it's hard to build consensus when the direction of an existing discussion that has made significant progress is changed quickly, according to one comment. That can lead to the direction going back and forward, never pleasing anyone. Live & learn, Dan.
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > > It seems to me we are very very close to having something people are > happy with. If I were to change the wording > > FROM > > "If the major version of the jars loaded in a JVM differ, the user > will need to separate the two versions by using separate classloaders. " > > TO > > "If the major version of the jars loaded in a JVM differ, the user may > need to separate the two versions by using separate classloaders. If > an incompatibility exists, it will be clearly documented. Any > undocumented incompatibilities will be treated as a bug." > > would anyone be unhappy with this? PLEASE let me know by Friday. If > I don't hear any objections I will put it up for (sigh) another vote. > I like this wording better than the other and wouldn't veto it, but I am fully willing to disclose that I couldn't offer it my binding +1 either because I think, to use Naka's term, it is a trap. I don't really believe that there will be no user impact but will hold you to it. I woke up this morning and realized I don't know how I would fix a bug in shared code with mixed jars, because the class in the other jar would mask my fix. I'm sure you have something planned for that and you don't have to explain it to me now, but when you submit your patch, the very first thing I'll do is 1) try to fix an imaginary customer bug and put the unchanged jar first 2) try to add a parameter to a message with no impact to the old jars regardless of classpath. Those should both be doable with what you have planned right? Kathey
Re: VOTE: Principles of sharing code
Kathey Marsden wrote: I like this wording better than the other and wouldn't veto it, but I am fully willing to disclose that I couldn't offer it my binding +1 either because I think, to use Naka's term, it is a trap. I don't really believe that there will be no user impact but will hold you to it. OK, fair enough. I appreciate your rigor and concern. I woke up this morning and realized I don't know how I would fix a bug in shared code with mixed jars, because the class in the other jar would mask my fix. I'm sure you have something planned for that and you don't have to explain it to me now, but when you submit your patch, the very first thing I'll do is 1) try to fix an imaginary customer bug and put the unchanged jar first 2) try to add a parameter to a message with no impact to the old jars regardless of classpath. Those should both be doable with what you have planned right? Yes, they should be. Let me try to answer them here and see what you think. Scenario 1 - Fixing a Bug If I have it right the issue of "masking" comes up only when you have a mixed version environment. Let's say the user is running with the 10.2 embedded driver and the 10.3 client driver. Hopefully the user tells us "I'm using both 10.2 and 10.3." Then we would fix the bug in both the 10.2 and 10.3 common code. If for some reason the user doesn't tell us this, we may fix the bug in the wrong version the first time, but after getting more info from the customer it will be clear that the bugfix should be ported to the other branch. We could have a policy that a common bug fix should be ported to all releases currently available and supported, but that seems like overkill and against the policy of "scratch your own itch" -- a bug should be fixed in the versions that matter to the user who has the bug. So I would argue when you do your test, you should make the fix in both versions of the common code and you should see no problem. Scenario 2 - Adding a parameter to a message This is a great example, and I'm glad you brought it up, I hadn't thought of it. You've motivated me to work through as many use cases as possible and document these on the Wiki page as part of our "hashing out" of how to do common code. Anyway, here's how I would do it. This probably isn't obvious, but message strings should be viewed as interfaces that should follow the same compatibility rules as Java methods. I'll add this note on the Wiki page. If you change the "signature" of a message, this is an incompatible change, and you should create a new message rather than modify the existing message. Since it's likely the SQL State is the same for the old and new messages, you need to deal with this. Let's say the old message id is "XJ786.S". The new message id should be something like "XJ786.S.2". To support forward-compatibility, you need to add a feature id like CommonInfo.SQLSTATE_MSGID_XJ786_2, and the code can check to see if the feature exists. If it doesn't, it uses the old message. Alternately, the messaging infrastructure knows how to build a "default message" if the message id can't be found, and you can just take advantage of that. But this default message is pretty ugly and IMHO should be avoided. David Kathey begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > > > Scenario 1 - Fixing a Bug > > If I have it right the issue of "masking" comes up only when you have > a mixed version environment. Let's say the user is running with the > 10.2 embedded driver and the 10.3 client driver. Hopefully the user > tells us "I'm using both 10.2 and 10.3." Then we would fix the bug in > both the 10.2 and 10.3 common code. > > If for some reason the user doesn't tell us this, we may fix the bug > in the wrong version the first time, but after getting more info from > the customer it will be clear that the bugfix should be ported to the > other branch. > > We could have a policy that a common bug fix should be ported to all > releases currently available and supported, but that seems like > overkill and against the policy of "scratch your own itch" -- a bug > should be fixed in the versions that matter to the user who has the bug. > > So I would argue when you do your test, you should make the fix in > both versions of the common code and you should see no problem. > > Now *that* is user impact, developer impact and support impact. The fact that the bug fix revision level will regress with mixed jars if the new jar is not first, is an important product behaviour change to mention. I don't fix a bug normally for a single end user. I fix a bug for a product that is deployed (hopefully) to thousands of users. You are saying I need to know the configuration of all of those sites and deal with them one by one. It's impossible. And this is not 10.2 and 10.3 mixing that we are talking about. Installing some product with the embedded driver at 10.2.1.1 might mask a client install at 10.2.1.19 and make something that is working just fine break in strange ways, where as now the two are totally independent. Kathey
Re: VOTE: Principles of sharing code
Scenario 1 - Fixing a Bug If I have it right the issue of "masking" comes up only when you have a mixed version environment. Let's say the user is running with the 10.2 embedded driver and the 10.3 client driver. Hopefully the user tells us "I'm using both 10.2 and 10.3." Then we would fix the bug in both the 10.2 and 10.3 common code. If for some reason the user doesn't tell us this, we may fix the bug in the wrong version the first time, but after getting more info from the customer it will be clear that the bugfix should be ported to the other branch. We could have a policy that a common bug fix should be ported to all releases currently available and supported, but that seems like overkill and against the policy of "scratch your own itch" -- a bug should be fixed in the versions that matter to the user who has the bug. So I would argue when you do your test, you should make the fix in both versions of the common code and you should see no problem. Now *that* is user impact, developer impact and support impact. The fact that the bug fix revision level will regress with mixed jars if the new jar is not first, is an important product behaviour change to mention. I don't fix a bug normally for a single end user. I fix a bug for a product that is deployed (hopefully) to thousands of users. You are saying I need to know the configuration of all of those sites and deal with them one by one. It's impossible. And this is not 10.2 and 10.3 mixing that we are talking about. Installing some product with the embedded driver at 10.2.1.1 might mask a client install at 10.2.1.19 and make something that is working just fine break in strange ways, where as now the two are totally independent. I have lost the track of this argument. I am confused by the phrase "the bug fix revision level will regress with mixed jars". Can you help me understand how behavior regresses, that is, becomes worse? I'm only seeing two cases here: o The bug fix is applied to the wrong Derby version (the second one in the classpath), in which case it is a NOP. The experience for the customer will be that the bug was not fixed. However, no new regressions are introduced. o The bug fix is applied to the correct Derby version (the first one in the classpath). This fixes the bug for the customer. But what behavior is made worse here? A concrete example would help. Thanks.
Re: VOTE: Principles of sharing code
I had to smile, your statement "now *that* is user impact" sounded like Perry Mason nailing a defense witness. I think you have caught me -- there is some user impact -- but less than you imply, which I will discuss below. I stand by my belief that sharing code has enough value to merit us dealing with this. From supporting customers in an environment with duplicated code, I can tell you that you have exactly the same problem you are describing here. It is so easy for a developer to fix a bug in copy A of some code but forget to fix it in copy B. The end user experience is the same -- I thought you fixed this bug, but you didn't! You also avoid strange behavior such as slightly different SQL States between client and server, or a slightly different treatment of a data type, or slightly different driver behavior, or the fact that JDBC feature A is supported in the embedded engine but not in the network client -- all things we are trying to solve. The power of leverage and productivity that you get with common code is also significant. A developer need only build and test functionality once, rather than twice or even three times (we have I think at least four different implementations of internationalization code in our code base -- tools, client, network server and engine). What about when we want to add a logging infrastructure, or monitoring, or...? Are we going to have two or three copies of these as well? After careful consideration, I have come to the conclusion that I also am not willing to create a strange and somewhat disturbing development environment through cloning classes and those sorts of solutions we have looked at in the past. In terms of your particular issue, I'd like to challenge that it is as bad as you say it is. Why would someone want to run at different patch versions of the same minor version in the same VM (as in your example, 10.2.1.1 masking 10.2.1.19)? Maybe I'm missing something, but isn't the point of mixed versions to support running with new/old functionality, which doesn't happen between patch or maintenance releases? So it seems you would see at a minimum different minor versions (e.g. 10.2 and 10.3) in a mixed version environment. So, as I see it, this can be solved in one of two ways: Solution A - Port the Fix - Have a policy that any bug fix to common code must be ported to the latest patch branches of the same minor revision where the bug was found (e.g. the bugfix should be ported to 10.2.0.1, 10.3.3.2, 10.4.0.1) - The user in a mixed environment must get the latest patch release for each revision they are using. They are likely doing this anyway. Yes, I agree, this a change in our requirements and is thus arguably user impact (e.g. a change in how our user experiences our product, although I think I agree with Rick this is not a regression). - A user running in a mixed environment with two major revisions may encounter that the bug was not fixed after a patch, but this should be fairly easy to resolve. Again, a little user impact, but again not what I'd consider a regression. Solution B - Separate Out the Common Code You do not have this problem if the code common between the network client and engine is placed into a separate jar file. Yes, again, user impact, but not a regression. According to the Jakarta Commons versioning guidelines that we have proposed to follow, this is an "interface-compatible change" and only requires a new point release (e.g. from 10.1 to 10.2). See http://jakarta.apache.org/commons/releases/versioning.html One could raise the concern that code shared between network client and network server should be in a different jar file so that the embedded footprint doesn't grow unnecessarily. But I don't see where you would ever run a different version of the network server and the network client in the same VM. So if there is code that is common between the network server and the network client, I argue that that can safely be embedded in derbyclient.jar and derbynet.jar. I can go either way, but my preference is toward Solution B. This has a one-time user impact (and not a very large one) and we don't have to worry about users being surprised that a bug wasn't fixed when they upgraded one version but not the other. Thanks, David Kathey Marsden wrote: David W. Van Couvering wrote: Scenario 1 - Fixing a Bug If I have it right the issue of "masking" comes up only when you have a mixed version environment. Let's say the user is running with the 10.2 embedded driver and the 10.3 client driver. Hopefully the user tells us "I'm using both 10.2 and 10.3." Then we would fix the bug in both the 10.2 and 10.3 common code. If for some reason the user doesn't tell us this, we may fix the bug in the wrong version the first time, but after getting more info from the customer it will be clear that the bugfix should be ported to the other branch. We co
Re: VOTE: Principles of sharing code
Rick Hillegas wrote: > >>> Scenario 1 - Fixing a Bug >>> >>> If I have it right the issue of "masking" comes up only when you have >>> a mixed version environment. Let's say the user is running with the >>> 10.2 embedded driver and the 10.3 client driver. Hopefully the user >>> tells us "I'm using both 10.2 and 10.3." Then we would fix the bug in >>> both the 10.2 and 10.3 common code. >>> >>> If for some reason the user doesn't tell us this, we may fix the bug >>> in the wrong version the first time, but after getting more info from >>> the customer it will be clear that the bugfix should be ported to the >>> other branch. >>> >>> We could have a policy that a common bug fix should be ported to all >>> releases currently available and supported, but that seems like >>> overkill and against the policy of "scratch your own itch" -- a bug >>> should be fixed in the versions that matter to the user who has the >>> bug. >>> >>> So I would argue when you do your test, you should make the fix in >>> both versions of the common code and you should see no problem. >>> >>> >>> >> >> >> Now *that* is user impact, developer impact and support impact. The >> fact that the bug fix revision level will regress with mixed jars if the >> new jar is not first, is an important product behaviour change to >> mention. I don't fix a bug normally for a single end user. I fix a bug >> for a product that is deployed (hopefully) to thousands of users. You >> are saying I need to know the configuration of all of those sites and >> deal with them one by one. It's impossible. And this is not 10.2 and >> 10.3 mixing that we are talking about. Installing some product with the >> embedded driver at 10.2.1.1 might mask a client install at 10.2.1.19 >> and make something that is working just fine break in strange ways, >> where as now the two are totally independent. >> > I have lost the track of this argument. I am confused by the phrase > "the bug fix revision level will regress with mixed jars". Can you > help me understand how behavior regresses, that is, becomes worse? I'm > only seeing two cases here: > > o The bug fix is applied to the wrong Derby version (the second one in > the classpath), in which case it is a NOP. The experience for the > customer will be that the bug was not fixed. However, no new > regressions are introduced. > > o The bug fix is applied to the correct Derby version (the first one > in the classpath). This fixes the bug for the customer. But what > behavior is made worse here? > > A concrete example would help. Thanks. > > This is the circumstance that Dan was talking about where jars are mixed by circumstances. I quote Dan's July 13 (Wow) JIRA-289 comment. Dan said . >In case it's not clear, this is the type of situation I'm talking about. >Application A is a client server application and requires Derby client 10.2 >Application B is a local embedded Derby application and requires Derby engine 10.3 >Now I want to run a client to A and B in the same JVM for some reason, maybe portlets, maybe multiple applications >in the same app server, maybe application B requires access to application A. >This would be supported with the current model, complete flexibility because the code bases are separate. >So do we want to lose this flexibility, or support it with the shared code model? Now lets change the versions a bit and say our classpath is in this order derby.jar;derbyclient.jar Application A is at 10.2.1.2 - They just last week got a critical bug fix in shared code and are happy to be running again. Application B is at 10.1.2.1- They are running fine too but planning to upgrade next week. next week rolls around and the user installs their CD from Application B which contains 10.2.1.1 . Application B is working great, but because the new derby.jar masks the fix in derbyclient.jar, Application A is broken again. I agree that jar mixing is an unfortunate reality, but it is a reality and has to stay safe and legal at least for a period of time while work is done to make it rare. Kathey
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > You also avoid strange behavior such as slightly different SQL States > between client and server, There is absolutely nothing stopping you from using the same source property files that we have for server for client messages. They aren't java files so don't have the same problems of cloning and can let you get the messages localized without all this controversy I think. > In terms of your particular issue, I'd like to challenge that it is as > bad as you say it is. > > Why would someone want to run at different patch versions of the same > minor version in the same VM See my response to Rick. Nobody wants it they just get it. It is one totally unrelated appllication affecting another. We need, I think, to discourage jar mixing by changing the documentation to encourage application developers to use classloaders to insulate their application and protect others from it, printing warnings to the log file when jars are mixed or there is more than one copy or version of derby in the classpath, providing a way to print sysinfo to the log when classes have been loaded with classloaders and possibly even deprecating the ability to mix jars if the community wills it because of the impact on development, but it can't happen overnight like this. It is an application developer education issue and even those that understand the need for classloaders now need some notice. > So, as I see it, this can be solved in one of two ways: > > Solution A - Port the Fix > Solution B - Separate Out the Common Code > > You do not have this problem if the code common between the network > client and engine is placed into a separate jar file. Neither of these work for the scenario I mentioned in reply to Rick's mail. Application A still inadvertently regresses and the user has to change their classpath to fix it. (But of course they first have to figure out that is what they have to do which is the hard part) Kathey
Re: VOTE: Principles of sharing code
Kathey Marsden wrote: > We need, I think, to discourage jar mixing by changing the > documentation to encourage application developers to use classloaders to > insulate their application and protect others from it, -1 : Derby needs to be easy to use, requiring developers to use class loaders is too much. We must follow the standard JDBC driver/data source paradigm, not have any additional requirements. If it's a problem, then Derby should handle it internally. (no idea how, I just talking general principles here). > printing warnings > to the log file when jars are mixed +1 : I do think Derby should refuse to boot if the network server jar does not match the embedded jar. > or there is more than one copy or > version of derby in the classpath, -0 : hard to rely on, since class path is not the only loading mechanism. providing a way to print sysinfo to > the log when classes have been loaded with classloaders +1 - some stored procedure version of sysinfo would be good, maybe even one that doesn't require a database. E.g. a sysinfo ping to the network server could report version info. Dan.
Re: VOTE: Principles of sharing code
Thanks, Kathey. I still don't understand the scenario you're describing. It seems to me that application A never saw the bug fix at all because the shared code was always shadowed by the version which application B needed. However, the following scenario could occur and might be akin to the problem which is troubling you: o Two applications, EmbeddedApp and ClientServerApp run in the same VM. o EmbeddedApp contains derby.jar(rev 1) o ClientServerApp contains derbyclient.jar(rev 3) o classpath=EmbeddedApp;ClientServerApp o ClientServerApp wants an important bug fix in shared code available in Derby rev 4. However, after upgrading to derbyclient.jar(rev 4), ClientServerApp still can't enjoy the bugfix because the improved shared code is shadowd by the old, buggy version in EmbeddedApp. The only way that ClientServerApp can enjoy this bugfix is if EmbeddedApp upgrades to use derby.jar(rev 4). It's worth pointing out that this problem already exists today: o Two applications, ClientServerApp1 and ClientServerApp2 run in the same VM. o ClientServerApp1 contains derbyclient.jar(rev 1) o ClientServerApp2 contains derbyclient.jar(rev 3) o classpath=ClientServerApp1;ClientServerApp2 o ClientServerApp2 wants an important bug fix in shared code available in Derby rev 4. However, after upgrading to derbyclient.jar(rev 4), ClientServerApp2 still can't enjoy the bugfix because the improved code is shadowd by the old, buggy version in ClientServerApp1. The only way that ClientServerApp2 can enjoy this bugfix is if ClientServerApp1 upgrades to use derbyclient.jar(rev 4). The authority which owns this VM and classpath has to coordinate the two applications. Our users are already exposed to this problem. I think that the benefits of code sharing are significant enough that they warrant slightly increasing this exposure. Regards, -Rick Kathey Marsden wrote: Rick Hillegas wrote: Scenario 1 - Fixing a Bug If I have it right the issue of "masking" comes up only when you have a mixed version environment. Let's say the user is running with the 10.2 embedded driver and the 10.3 client driver. Hopefully the user tells us "I'm using both 10.2 and 10.3." Then we would fix the bug in both the 10.2 and 10.3 common code. If for some reason the user doesn't tell us this, we may fix the bug in the wrong version the first time, but after getting more info from the customer it will be clear that the bugfix should be ported to the other branch. We could have a policy that a common bug fix should be ported to all releases currently available and supported, but that seems like overkill and against the policy of "scratch your own itch" -- a bug should be fixed in the versions that matter to the user who has the bug. So I would argue when you do your test, you should make the fix in both versions of the common code and you should see no problem. Now *that* is user impact, developer impact and support impact. The fact that the bug fix revision level will regress with mixed jars if the new jar is not first, is an important product behaviour change to mention. I don't fix a bug normally for a single end user. I fix a bug for a product that is deployed (hopefully) to thousands of users. You are saying I need to know the configuration of all of those sites and deal with them one by one. It's impossible. And this is not 10.2 and 10.3 mixing that we are talking about. Installing some product with the embedded driver at 10.2.1.1 might mask a client install at 10.2.1.19 and make something that is working just fine break in strange ways, where as now the two are totally independent. I have lost the track of this argument. I am confused by the phrase "the bug fix revision level will regress with mixed jars". Can you help me understand how behavior regresses, that is, becomes worse? I'm only seeing two cases here: o The bug fix is applied to the wrong Derby version (the second one in the classpath), in which case it is a NOP. The experience for the customer will be that the bug was not fixed. However, no new regressions are introduced. o The bug fix is applied to the correct Derby version (the first one in the classpath). This fixes the bug for the customer. But what behavior is made worse here? A concrete example would help. Thanks. This is the circumstance that Dan was talking about where jars are mixed by circumstances. I quote Dan's July 13 (Wow) JIRA-289 comment. Dan said . In case it's not clear, this is the type of situation I'm talking about. Application A is a client server application and requires Derby client 10.2 Application B is a local embedded Derby application and requires Derby engine 10.3 Now I want to run a client to A and B in the same JVM for some reason, maybe portlets, maybe multiple applications in the same app server, maybe application B requires access to application A. This would be
Re: VOTE: Principles of sharing code
Rick Hillegas wrote: > Thanks, Kathey. I still don't understand the scenario you're describing. > It seems to me that application A never saw the bug fix at all because > the shared code was always shadowed by the version which application B > needed. However, the following scenario could occur and might be akin to > the problem which is troubling you: > > o Two applications, EmbeddedApp and ClientServerApp run in the same VM. > > o EmbeddedApp contains derby.jar(rev 1) > > o ClientServerApp contains derbyclient.jar(rev 3) > > o classpath=EmbeddedApp;ClientServerApp > > o ClientServerApp wants an important bug fix in shared code available in > Derby rev 4. However, after upgrading to derbyclient.jar(rev 4), > ClientServerApp still can't enjoy the bugfix because the improved shared > code is shadowd by the old, buggy version in EmbeddedApp. The only way > that ClientServerApp can enjoy this bugfix is if EmbeddedApp upgrades to > use derby.jar(rev 4). > > It's worth pointing out that this problem already exists today: > > o Two applications, ClientServerApp1 and ClientServerApp2 run in the > same VM. > > o ClientServerApp1 contains derbyclient.jar(rev 1) > > o ClientServerApp2 contains derbyclient.jar(rev 3) Well, you did a sneaky switch to two versions of the client jar! The problem does not exist today if you had kept the example the same (derby.jar and derbyclient.jar) and shared code did not exist. I don't think anyone is saying that shared code has to fix every mismatch, but the real point is, do we want to (potentially) regress in this area? Dan.
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > One could raise the concern that code shared between network client and > network server should be in a different jar file so that the embedded > footprint doesn't grow unnecessarily. But I don't see where you would > ever run a different version of the network server and the network > client in the same VM. So if there is code that is common between the > network server and the network client, I argue that that can safely be > embedded in derbyclient.jar and derbynet.jar. Running the client and server at different versions within a jvm is what the whole dicussion has been about. Or, maybe, the example has been the client and the embedded engine, but I've been assuming that meant the server as well. This: client (v1) for application A embedded (v3) for application B is the same as client(v1) for application A embedded (v3) + server (v3) for application B for the purposes of this discussion. Dan.
Re: VOTE: Principles of sharing code
Kathey Marsden wrote: David W. Van Couvering wrote: You also avoid strange behavior such as slightly different SQL States between client and server, There is absolutely nothing stopping you from using the same source property files that we have for server for client messages. They aren't java files so don't have the same problems of cloning and can let you get the messages localized without all this controversy I think. That's actually not true. The server-side messages are *huge* and are currently not available in derbyclient.jar. Adding them would cause major client bloat (on disk if not at runtime), and only a small percentage of them are shareable. This problem isn't solvable by cutting/pasting code. If I don't want client bloat, I'd have cut/paste all shared messages *and* their translations. It makes much more sense to migrate shared messages to a common location. This requires a shared code infrastructure. In terms of your particular issue, I'd like to challenge that it is as bad as you say it is. Why would someone want to run at different patch versions of the same minor version in the same VM See my response to Rick. Nobody wants it they just get it. It is one totally unrelated appllication affecting another. I want to make sure I understand. I think you're saying that there can be situations where there is a VM which gets its classes in some untransparent way. For example, Application A puts its classes in one directory and Application B puts its jars in another directory, and then the VM is configured (somehow) to load classes from the union of these directories. I'm sure there are other examples, but this is the one that comes to mind. The common thread here is there is an application hosting environment where multiple applications are deployed into the same VM, and if the same jar files are deployed into both applications, then you can not guarantee that a particular instance of a class within these duplicated jars will actually be loaded at runtime, unless you carefully manage your classloading order or using separate classloaders. It seems to me that in such an environment Derby already has this problem. If Application A is running an embedded database and Application B is running an embedded database, and the Derby jar files for A precede B in the classloading order, then a bug fix in the engine code for B will be shadowed by the code in A. The same is true if two applications are running with the network client and have a separate set of Derby jar files. So what shared code is introducing is an incremental increase in the environments where such problems already exist, where one application is running with the network client driver and another is running the embedded driver. It would seem to me the situation where there are two applications both running embedded would be more likely than the network/embedded case. Yet none of us seem to have ever heard anyone complaining that they have encountered this shadowing problem (or I'm sure it would have come up in the context of this discussion). My suspicion is that this is because such an environment is extremely rare, if it exists at all. All application hosting environments I know of use classloaders, and this type of shadowing issue is one of the primary reasons for this. At any rate, my main point is I don't think that shared code is introducing a major regression to the product. We need, I think, to discourage jar mixing by changing the documentation to encourage application developers to use classloaders to insulate their application and protect others from it, printing warnings to the log file when jars are mixed or there is more than one copy or version of derby in the classpath, providing a way to print sysinfo to the log when classes have been loaded with classloaders and possibly even deprecating the ability to mix jars if the community wills it because of the impact on development, but it can't happen overnight like this. It is an application developer education issue and even those that understand the need for classloaders now need some notice. I would be happy to write some information to the log so people know they have mixed versions. This would help with the problem that already exists, and will continue to help when we have shared code. To your point about "not doing this overnight," I suspect what you're saying is "warn people about it in 10.2 and then do it in 10.3". You also seem to indicate that you feel all the needs for shared code are "future needs." I'm not convinced of that. I think that, particularly as the community continues to grow, there is a timely need for this that really can't wait another year (assuming six month cycles for minor releases) before it sees the light of day. I am sure I am not the only one who is working on a feature where shared code would help or is needed. I know Rick wants
Re: VOTE: Principles of sharing code
Well, you did a sneaky switch to two versions of the client jar! The problem does not exist today if you had kept the example the same (derby.jar and derbyclient.jar) and shared code did not exist. Right. This is the small incremental exposure introduced by David's proposal. I don't think anyone is saying that shared code has to fix every mismatch, but the real point is, do we want to (potentially) regress in this area? Dan. Right. David's proposal doesn't claim to fix this problem. And his proposal does introduce some extra exposure. I'm arguing that the extra exposure is small compared to the current exposure. I'm also arguing that the benefits of his proposal are worth this extra exposure. I hope we're not talking past one another here. It's so easy to do in email. I think we may agree on the following points: o There is an existing problem when you mix two Derby-enabled applications in the same vm. o David's proposal increases our customer's exposure to this problem. However, we may part ways on the following points: o How significant that extra exposure is. o Whether the benefits of code sharing justify that extra exposure. Regards, -Rick
Re: VOTE: Principles of sharing code
Rick Hillegas wrote: > Thanks, Kathey. I still don't understand the scenario you're > describing. It seems to me that application A never saw the bug fix at > all because the shared code was always shadowed by the version which > application B needed. Because for 10.1 there is no shared code, the Application B 10.1 derby.jar and the Application A 10.2 derbyclient.jar were able to coexist totally independently.Upgrading Application B to 10.2 with the shared code caused the regression in Application A.It is a very subtle trap but very dramatic in its effect. > It's worth pointing out that this problem already exists today: > It is true that there are problems in this regard today with multiple embedded apps or multiple client apps. There is almost always a jar mixing issue in the support queue for jar conflicts and they are ugly. The one we are working on right now (with an older version of Cloudscape) is not just a matter of let's find all the duplicate jars we can and replace them, but includes necessary patches to some of the products involved. I personally am not keen on exacerbating the problem. That's perhaps just selfishness because I get to deal with it all the time, but I hope you can agree that the scope of user impact in the proposal is incorrect. It should be clear that we are regressing the ability to mix the client and server jars. Kathey
Re: VOTE: Principles of sharing code
Interesting point Dan - However we also have to weigh what code sharing brings to development and maintainability of Derby and especially in the long run versus the patch shadowing issue in particular application environments which does *not* seem to me like the most majority of running configurations today (being more like corner case ones) or even in the future (95/5 rule let's say) I could be wrong of course - I'm already seeing some discrepancies in the codeline with pieces of code containing similar logic but where changes (patches) have been applied to one side and not the other - Yes this is not supposed to happen but it does happen and I think it is bound to happen even more within a community (as more contributors come onboard). So, not providing code sharing can also lead to not propagating code changes in various places throughout Derby where similar piece(s) of logic have been duplicated - I also believe that the redundancy of piece of logic can only get worse along the years if there is no code sharing (since we would keep on adding redundant pieces of logic between the client and embedded engine instead of sharing whenever it makes sense). I understand the issue with patch shadowing and I think it is a valid one but when I look at it at not being the majority of applications running Derby out there (I may be wrong again). We could rather document on how to address some *potential* issue that might arise with a 5% of running configuratoins out there versus sacrificing maintainability of Derby in the long run. But this is *only* my personal point of view from my own perspective and context - Support might have a different point of view but code sharing also aims at improving quality of the code in the long run IMHO which will benefit Derby users as well.. Cheers, --francoisOn 10/28/05, Daniel John Debrunner <[EMAIL PROTECTED]> wrote: I don't think anyone is saying that shared code has to fix everymismatch, but the real point is, do we want to (potentially) regress inthis area?Dan.
Re: VOTE: Principles of sharing code
Rick Hillegas wrote: > I hope we're not talking past one another here. It's so easy to do in > email. I think we may agree on the following points: > > o There is an existing problem when you mix two Derby-enabled > applications in the same vm. > > o David's proposal increases our customer's exposure to this problem. > > However, we may part ways on the following points: > > o How significant that extra exposure is. > > o Whether the benefits of code sharing justify that extra exposure. That's a good summary. I thought the last set of principles (the ones without the 'must use classloader') took a good approach to minimize the exposure. I think we should move forward on shared code, I think we all know it's needed at some point. If we reject shared code now, then how do we get to text indexing, xpath/xquery etc. I don't want to waste my time writing text search code when Lucene exists. I think the principles are good, especially this one: 'This implementation will not have any significant impact on jar file size or otherwise affect product distribution or usage. ' though, probably it should say 'Any implementation will ...', or 'Code sharing will ...' since the vote is on the principles, not an implementation. I think though, that we do need to address Kathey's concerns, her ideas about logging different versions etc. are all useful. And we do need to be ever vigilant on the usability of Derby. Let's have some shared code, so that we can see what problems it creates and then solve them. Most likely any code we do affects usability in some way, and in some cases we have to analyze the risk and see if the value out-weighs the risk. Dan. PS. this post was brought to you by too much Peet's coffee in the afternoon ...
Re: VOTE: Principles of sharing code
Thanks, Dan. I would like to know if we need to have a vote on the overall principles of shared code or if people just want to see an examlpe implementation. I am fine either way, although personally I'd like to make sure we agree on the principles before I spend too much time on an implementation. I would say if I see at least two committers besides me saying they want a vote on the principles, then I will send out a proposed wording, and then we can discuss and do a vote. Otherwise, I will work on an initial implementation with some small set of messages migrated over to the shared component environment that demonstrates the framework, and submit *that* for a vote. Thanks, David Daniel John Debrunner wrote: Rick Hillegas wrote: I hope we're not talking past one another here. It's so easy to do in email. I think we may agree on the following points: o There is an existing problem when you mix two Derby-enabled applications in the same vm. o David's proposal increases our customer's exposure to this problem. However, we may part ways on the following points: o How significant that extra exposure is. o Whether the benefits of code sharing justify that extra exposure. That's a good summary. I thought the last set of principles (the ones without the 'must use classloader') took a good approach to minimize the exposure. I think we should move forward on shared code, I think we all know it's needed at some point. If we reject shared code now, then how do we get to text indexing, xpath/xquery etc. I don't want to waste my time writing text search code when Lucene exists. I think the principles are good, especially this one: 'This implementation will not have any significant impact on jar file size or otherwise affect product distribution or usage. ' though, probably it should say 'Any implementation will ...', or 'Code sharing will ...' since the vote is on the principles, not an implementation. I think though, that we do need to address Kathey's concerns, her ideas about logging different versions etc. are all useful. And we do need to be ever vigilant on the usability of Derby. Let's have some shared code, so that we can see what problems it creates and then solve them. Most likely any code we do affects usability in some way, and in some cases we have to analyze the risk and see if the value out-weighs the risk. Dan. PS. this post was brought to you by too much Peet's coffee in the afternoon ... begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > Thanks, Dan. I would like to know if we need to have a vote on the > overall principles of shared code or if people just want to see an > examlpe implementation. I am fine either way, although personally I'd > like to make sure we agree on the principles before I spend too much > time on an implementation. > I think the requirement for a vote, be it in principle or patch comes primarily from the resulting product behaviour changes. I think for the changes you have proposed, that means reduced support for mixing server and client jar versions. I think I finally understand the change in a way that I can explain it to users and support folks that are affected by it but know nothing about the internals of Derby, what's shared or how, and I think it goes like this: Behaviour Change: Derby will continue to allow mixing different versions of derbyclient.jar and derby.jar in the same JVM classpath but beginning with Derby 10.2 this capability will be affected by classpath order. If the lower revision level jar comes first in the classpath, new functionality and bug fixes may not be available, but functionality at the lower revision level will not be affected. Placing the higher revision level jar first in the classpath will make the functionality and bugfixes from the higher revision level available. Example: If a bug is fixed in version 10.2.1.0 (1235) and a single JVM has a client application with version 10.2.1.0 (1234) derbyclient.jar and an embedded application with 10.1.2.0 (1235) derby.jar in it's classpath, it may be necessary to either upgrade the client version to 10.1.2.0 (1235) or make sure derby.jar is first in the classpath for the embedded application to see the fix. Does this sound like an accurate description of the behaviour change from the user/support perspective?Is there anything else we should watch out for in environments where jar versions are mixed? Will derbytools.jar be affected? Just to be clear, I am not challenging anything here. I'm just trying to understand and make sure I explain this to others correctly and deal with it properly from a support perspective. Thanks Kathey
Re: VOTE: Principles of sharing code
This is correct, except I think limiting it to "classpath" may cause confusion in some environments. See for example Bryan's struggles because he had a Derby jar file in $JAVA_HOME/lib/ext, which takes precedence over the classpath. You might say something more general like "ensuring that the higher revision level jar is first in the order that classes are loaded (e.g. by placing it first in the classpath) will make the functionality and bugfixes from the higher revision level available" Also, it's possible that the higher revision functionality is still visible *if* it's from a package that doesn't exist in the older revision of the common code. For example, if you add org.apache.derby.common.foobar to revision 1234 and this package is not in revision 1233, then the classes in foobar will be visible regardless of classpath ordering. This is *not* the case if you add new classes to an existing package, as classes for a given package can not be loaded from multiple jar files if the packages are sealed (which I'm assuming is the case for common packages). Do I have that right? Also, you don't talk about upgrade scenarios, where the bugfix is there, but then someone upgrades and all of a sudden the bugfix is gone. This can *only* occur if (a) the bug fix is in a new package and thus visible even though the jar file doesn't have precedence and then (b) the upgrade places the jar file with most precedence at a revision level that has the new package but *not* with the bugfix. e.g.: version 1234 has a new package, common.foobar version 1233 does *not* have common.foobar jar ordering is 1233:1234 1234 is upgraded to 1234.1 which has a bug fix in common.foobar, so now you have 1233:1234.1 and all is good then 1233 is upgraded to 1234 and you have 1234:1234.1 and the bugfix "disappears" David Kathey Marsden wrote: David W. Van Couvering wrote: Thanks, Dan. I would like to know if we need to have a vote on the overall principles of shared code or if people just want to see an examlpe implementation. I am fine either way, although personally I'd like to make sure we agree on the principles before I spend too much time on an implementation. I think the requirement for a vote, be it in principle or patch comes primarily from the resulting product behaviour changes. I think for the changes you have proposed, that means reduced support for mixing server and client jar versions. I think I finally understand the change in a way that I can explain it to users and support folks that are affected by it but know nothing about the internals of Derby, what's shared or how, and I think it goes like this: Behaviour Change: Derby will continue to allow mixing different versions of derbyclient.jar and derby.jar in the same JVM classpath but beginning with Derby 10.2 this capability will be affected by classpath order. If the lower revision level jar comes first in the classpath, new functionality and bug fixes may not be available, but functionality at the lower revision level will not be affected. Placing the higher revision level jar first in the classpath will make the functionality and bugfixes from the higher revision level available. Example: If a bug is fixed in version 10.2.1.0 (1235) and a single JVM has a client application with version 10.2.1.0 (1234) derbyclient.jar and an embedded application with 10.1.2.0 (1235) derby.jar in it's classpath, it may be necessary to either upgrade the client version to 10.1.2.0 (1235) or make sure derby.jar is first in the classpath for the embedded application to see the fix. Does this sound like an accurate description of the behaviour change from the user/support perspective?Is there anything else we should watch out for in environments where jar versions are mixed? Will derbytools.jar be affected? Just to be clear, I am not challenging anything here. I'm just trying to understand and make sure I explain this to others correctly and deal with it properly from a support perspective. Thanks Kathey begin:vcard fn:David W Van Couvering n:Van Couvering;David W org:Sun Microsystems, Inc.;Database Technology Group email;internet:[EMAIL PROTECTED] title:Senior Staff Software Engineer tel;work:510-550-6819 tel;cell:510-684-7281 x-mozilla-html:TRUE version:2.1 end:vcard
Re: VOTE: Principles of sharing code
David W. Van Couvering wrote: > This is correct, except I think limiting it to "classpath" may cause > confusion in some environments. See for example Bryan's struggles > because he had a Derby jar file in $JAVA_HOME/lib/ext, which takes > precedence over the classpath. > > You might say something more general like "ensuring that the higher > revision level jar is first in the order that classes are loaded (e.g. > by placing it first in the classpath) will make the functionality and > bugfixes from the higher revision level available" > good point > Also, it's possible that the higher revision functionality is still > visible *if* it's from a package that doesn't exist in the older > revision of the common code. Since most users are not really up on what classes exist in the Derby implementation, I just punted with "may not be available" as in don't bank on it. > Also, you don't talk about upgrade scenarios, where the bugfix is > there, but then someone upgrades and all of a sudden the bugfix is gone. Yes, that one is a bit scary and worthwhile making people aware that it might happen, but assuming this change is approved, it mostly seems to me I need to get folks focused on the immediate workaround which is to get the highest rev jar loaded first, and hopefully make headway on the long term application isolation education issue which as Rick mentioned is already an issue (albeit a less mysterious one) for multiple embedded applications/components running in the same JVM.But first I better educate myself on class loaders, which apparently won't be a small task. http://mail-archives.apache.org/mod_mbox/www-women/200510.mbox/[EMAIL PROTECTED] Thanks for updating the user impact on the Wiki page http://wiki.apache.org/db-derby/SharedComponentVersioningGuidelines. I think the description of user impact, while less attractive is much more accurate now and I can believe it. Kathey
