On 25-Dec-08, at 4:27 PM, Ralph Goers wrote:
Jason,
I'm surprised you don't remember this discussion since we've had it
so many times.
The only thing I wanted to make sure you weren't doing was passing on
requirements as provisions transitively. If you know RPMs then we're
on the same wavelength. It was one of your previous emails that made
me wonder.
Again I don't think we're off to far from what you want Suse and most
of the other distributions employ a SAT solver to calculate a set of
suitable artifacts. The library that we are using, SAT4J, is capable
of doing this and is moreso used like this in P2 which is the update
manager technology used in Eclipse. It takes into account attributes
like platform and that's taken into account when doing resolution for
a particular target platform. But the first thing we need to do with
Mercury is make it work for what we do currently.
My opinion is that versions are just one piece of metadata, and a
not very important one at that since they are almost completely
arbitrary. When I speak of provides and requires I am referring to
exactly what RPMs do. Yes, every RPM has a version but that is not
actually what the RPM system uses to determine what can be
installed. If you look at an RPM you will see provides and requires.
A lot of folks think that these are used for package version
checking, but that is not completely true. The general format is
token [operator version]
Usually this might be for the form libstdc++ >= 2.0. But the
"tokens" are not limited to package names. They can be anything. For
example, xerces-impl might list as what it provides:
xerces-impl = 2.9.1
jaxp = 1.2,1.3
xml = 1.0,1.1
xml-schema=1.0
sax = 2.0.2
xinclude = 1.0
Debian packages handle this a little differently (see section 7.5 at http://www.debian.org/doc/debian-policy/ch-relationships.html)
. There the provides is different in that you cannot specify a
version so the above would have to be specified as
jaxp-1.2
jaxp-1.3
xml-1.0
xml-1.1
xml-schema-1.0
sax-2.0.2
xinclude-1.0
Sure, these are exactly the types of things SAT is good for and is why
most of the package managers employ some form of SAT solver.
What I envision for Maven would be that the xerces pom might look
like:
<project>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<version>2.9.1</version>
<provides>
<feature name="jaxp">1.2,1.3</feature>
<feature name="xml">1.0,1,1</feature>
<feature name="xml-schema>1.0</feature>
<feature name="sax">2.0.2</feature>
<feature name="xinclude">1.0</feature>
</provides>
</project>
A component might only be interested in JAXP 1.3 and not really care
which version of Xerces provides it. It should be able to specify
something like
<dependency>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<requires>
<feature name="jaxp">1.2</feature>
</requires>
</dependency>
This would allow any version of xerces that provides jaxp 1.2 to be
selected.
This is one thing we could certainly do, but honestly something I
don't think is useful most of the time. I haven't had many instances
myself where I didn't care about the implementation of a particular
specification. If you really could guarantee not only API but non-
functional requirements like performance then possibly. It won't be
hard to do things like this with SAT4J. But I haven't deployed in any
sort of standard application server in a long time. So if you're
thinking to take everything that a platform might provide in order to
mesh that with what requirements your application needs then sure. And
something like this is totally possible with Mercury and our use of
SAT4J.
On the other hand, a different component might want both JAXP 1.3
and xinclude support. This would limit the number of versions that
could be selected but would still be resolvable since the latest
version supports all of the requirements.
Yah, I think we're a long way from this being practical and I imagine
we would be forging the way here again. OSGi can provide something
like the servlet api but I don't think they provide capability maps,
or feature sets as you've depicted them above. But OSGi is certainly
closer here if you can make the mental jump of mapping a capability to
a package like javax.servlet.
While this example for Xerces might seem out of the ordinary, it
really isn't. If you look at commons-collections history page http://commons.apache.org/collections/history.html
it should be obvious that features could have easily been used to
describe the various changes made to the project as well as to
identify the binary incompatibilities that occurred.
Ralph, I honestly think again on larger scale it is not going to be a
technical problem. It's deciding on a global capability map that
everyone would use accurate. I mean we still argue with people about
versioning their dependencies and "why can't I just use a lib
directory of JARs" kind of mentality so while I think this is the
direction we should go it's use is going to limited for a while and
Mercury is just beginning its life.
Ralph
On Dec 25, 2008, at 8:10 AM, Jason van Zyl wrote:
On 25-Dec-08, at 2:02 AM, Ralph Goers wrote:
This whole thread makes my head hurt.
Frankly, and at the risk of being repetitious, I think all this
work into making this resolution based on versioning is pretty
much a waste of time.
I don't think anyone is OSGi land would agree with you. This is a
critical step along the way.
Now if your picture incorporated artifact metadata, such as
"provides" and "requires" attributes I'd sit up and take notice.
Until it does it really won't much matter how this works since
everything is going to continue to be locked down with managed
dependencies.
The whole point is to get down to a specified list of dependencies
that go into the runtime of your application. I don't disagree with
you here and Oleg certainly doesn't because we've discussed this at
length. What Mercury is provided is a way to arrive at that
possibility while along the way using the optimization functions to
get there. Otherwise it's a laborious manual process. We want to
aid this process during development with resolution and
visualization techniques. We don't want a system dynamically
resolving everything at runtime. The description of your
application may say it can be dynamic, but in practice you limit
what is resolved. To manually create your dependencyManagement, or
lock down list, is a complete pain in the ass.
I'm also not sure what you mean in your provides/requires model.
What you provide is something couched in terms your package. In
OSGi you generally export, or provide, packages in your library/
application you want to expose to others. What you import, or
require, are the dependencies your library/application needs. In
our model currently what a Maven project provides is itself as an
artifact. What it requires are its dependencies. This is how RPM
works. If you look in a spec file the list of what's provided is
usually itself. I'm not sure were on the same wavelength when
talking about requires/provides. A library can't provide anything
then the content of what it contains itself. In our course grained
model (or requires bundle in OSGi speak) it's the artifact you're
making.
All I'm really looking for is a) dependency resolution based on
artifact attributes, b) notification when an unresolvable conflict
exists, and c) a way to generate managed dependencies based on a
and b so that the build can always be repeated.
a) is definitely coming and it essentially looks like an LDAP query
for lack of anything else to compare it against
b) most definitely required and this is handled internally by what
we have and is going to be far easier to see given what Oleg has
c) exactly
We are not talking about different things. To resolve all your
dependencies from scratch at runtime would be ill advised. The
whole point of the system is so that we can resolve on demand when
changes are necessary. Oleg and I have talked for a long time about
the dynamic nature of OSGi and balancing that with the fact that
once your QA team approves something you better not start throwing
new bits into the mix or you'll invalidate what was tested.
Ultimately we want a list of pieces for an application that we know
don't change when we are at the end of the cycle and are ready to
ship. I'm pretty sure what you need is something we have accounted
for.
When building a standard web application there are really no
constraints or metadata establishing what your runtime versioning
looks like. So you can happily toss in some new JARs in your WEB-
INF/lib directory and if it all links at runtimes everything is
good. When you upgrade things maybe you add in a few JARs, or maybe
you rebuild the entire application and drop it into your container.
In OSGi there are constraints about what pieces can work together
at runtime. You do have the option of stating the exact version of
your requirements, but this is not generally how people setup their
applications. They specify ranges of versions that are acceptable.
This is not to say that you want the resolution to go hog wild at
runtime. This doesn't happen because people generally manage the
bundle repository -- which has no remote capabilities -- and they
drop new things in. So the runtime can dynamically accept new
versions of dependencies if a range allows it. What is resolved in
practice is limited by what's in the bundle repository.
So when Oleg is referencing all this resolution and the mechanics
of resolving it is toward the end of having a fixed list of
something that works and has been tested. We want the mechanics of
this to work in Mercury and be available in tooling like m2eclipse
and we're also working toward having a system where an application
can say "I will take anything you give me to resolve because I can
dynamically update myself" but it's going to be talking to Nexus
who will say back "yes, little application but I'm going to tell
you what you are going to use". To have to redeploy the metadata
with the artifacts of an application in order for it to resolve new
versions of application on N nodes of a large system is untenable.
You need to be able to let the application consume newer versions
of artifacts while at the same time limiting what it consumes. We
are attempting to shift all this management to Nexus where through
a virtual URL an application will resolve against a constrained set
of artifacts that might be available. In essence your fix Map of
artifacts available from dependencyManagement. So this is our end
goal but I don't think Mercury along way is incompatible with what
you require.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
A man enjoys his work when he understands the whole and when he
is responsible for the quality of the whole
-- Christopher Alexander, A Pattern Language
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Thanks,
Jason
----------------------------------------------------------
Jason van Zyl
Founder, Apache Maven
jason at sonatype dot com
----------------------------------------------------------
Our achievements speak for themselves. What we have to keep track
of are our failures, discouragements and doubts. We tend to forget
the past difficulties, the many false starts, and the painful
groping. We see our past achievements as the end result of a
clean forward thrust, and our present difficulties as
signs of decline and decay.
-- Eric Hoffer, Reflections on the Human Condition
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
