On 2011-06-19 19:02, Johannes Pfau wrote:
I still don't understand that completely. So does it list the files
which will be contained in the package later, or file dependencies
contained in other packages?
(I'm asking because I'm not familiar
with file-dependencies in package management systems. Most package
management systems make a package depend on other packages, but not on
the files in the packages)
Ok, let me explain. When developing a package management system the
first thing one has do decide is if the package should contain pre-built
binaries/libraries, we can call these binary packages, or the necessary
files to build the package when installing, we can call these source
package (not to be confused with the source type you've mentioned
below). As a third option, one could have a mixed package system
containing both binary and source packages. Maybe even mixed packages
could be possible.
When I first started thinking about Orbit I decided for source packages.
The reason for this is that the developer only have to create one
package or doesn't have to build the app/lib for all supported platforms
when releasing a new version of the package (although it would be good
to know that it works on all supported platforms).
When I first wrote down my ideas about Orbit on the wiki I was,
incorrectly, thinking it should have binary packages, hence the
confusion (even I'm confused). This was when the orbspec examples
contained the "imports" method. I then corrected this error and changed
"imports" to "files" to reflect that the packages are source packages.
Now, "files" contains all the necessary files to build the package when
installing it. These files most somehow be referenced in the orbspec
because the tool needs to know what files to put in the package. This is
for a package with no dependencies on other packages.
Note that usually you don't need to explicitly specify all the necessary
files, thanks to Ruby's fleshed out standard library. For example, you
could do like this:
imports Dir["**/*.d"]
Which will, recursively, include all *.d files in the current directory.
If you use Orbit together with Dake you don't have to specify any files
at all since Dake will know what files to include (this is noted in the
wiki).
If the package depends on other package this needs to be listed as well
in the orbspec. This is not showed in any of the examples on the wiki.
I hope this explains most of the things and I'm sorry for any confusion
I may have caused.
Another detail: I wouldn't use .orb for package extensions. We might
want to change the compression type later (tar+lzma for example),
so .orb.zip would be better. Then we could just use .orb.tar.xz with
the new compression. (This is also how archlinux works, for
example .pkg.tar.xz)
Ok, I can do that. I chose zip because that is available in Tango and
Phobos.
Yes, right now, zip is seems to be the best choice, but at some point
the small size difference between zip and lzma could matter.
It seems like C libraries would also be packaged with orb (the sqlite
example). This might be needed, but it will be a major pita for linux
packagers, as it'll likely cause conflicts. I think it should be
possible for those linux packages to hook into orb. Orb should
recognize something like 'orb --external libsqlite:library
--version 3.7.0' and then just assume that sqlite is installed (but
it should not assume that sqlites dependencies are installed - those
would have to be registered with --external again). This approach
should work well for D packages (so a D package is in Orb first, but
some distribution decides to package it. In this case they can add
the orb hooks to their packages). It's unlikely that a distribution
will change all C packages though. Probably at some time orb should
interact with 'pkg-config' to look for already installed C packages,
I'm not sure what's a good solution for this problem.
With the sqlite example, I was actually thinking about bindings. But
as you say, it would be good to be able to specify external
dependencies, like C libraries.
I totally forgot about bindings! I'm quite uncertain about packaging C
libs though: If we don't package C libs windows users
will have to acquire all C libs manually. But if we do provide C
libraries we have to decide if we also ship the C headers, etc. and it
will be some more maintenance work.
Yeah, I don't know exactly what to do with external dependencies. The
easiest would be to have it just as information for the user of the package.
'type :library' in the orbspec suggests that there'll be different
package types. I think this is a good idea so we don't have to use
package name hacks like 'libsqlite' 'libsqlite-dev' (debian)
Package types which make sense:
:doc --> documentation. Later possibly in a specific format?
:lib --> shared libraries (.so/.dll) when available
:slib --> static lib (.a/.lib)
:dev --> header files (.di)
:src --> source package used to build other packages
Yes, exactly.
Sound great. Orb could be the first package management system to get
that right. One more question is where .h headers and .di files would
go for C packages. Both in :dev or .h headers into additional :cdev
packages, or something like that.
I haven't though that far. The first step have to be to get regular, D
only, packages to work first.
We should also think about how the versioning scheme would interact
with git/hg/svn whatever snapshots and alpha/beta/rc releases. The
debian package system doesn't have explicit support for this which
leads to strange version numbers. Archlinux even uses different
packages for git versions (libsqlite-git) which also isn't a good
solution.
If you have any ideas I'm listening.
OK, my proposal follows, but be warned, it's a little longer than I
first thought :-)
Regarding alpha/beta/rc's a simple scheme could help:
all those releases are pre-releases, so consider a fictional libjson
as an example.
libjson 0.0.1 is released (a final release)
libjson 0.0.2 alpha1 is released --> prerelease 1
libjson 0.0.2 alpha2 is released --> prerelease 2
libjson 0.0.2 beta1 is released --> prerelease 3
libjson 0.0.2 beta2 is released --> prerelease 4
libjson 0.0.2 rc1 is released --> prerelease 5
libjson 0.0.2 is releases (final release)
So in this case 0.0.1< 0.0.2 pre1< 0.0.2 pre2< 0.0.2 pre3< 0.0.2
pre4< 0.0.2 pre5< 0.0.2 pre[X]< 0.0.2
The end user should specify for which packages he'd like to use
prereleases. A standard upgrade for libjson would look like this:
0.0.1 --> 0.0.2 so no prereleases should be installed be default.
If prereleases were enabled for that package, prereleases should be
upgraded automatically:
0.0.1 --> 0.0.2 pre1 --> 0.0.2 pre2 ... --> 0.0.2
It's also possible that a version 0.0.1.1 is released somewhere in
between:
libjson 0.0.1 is released (a final release)
libjson 0.0.2 alpha1 is released --> prerelease 1
libjson 0.0.1.1 rc1 is released --> prerelease 1
libjson 0.0.1.1 is released (a final release)
libjson 0.0.2 alpha2 is released --> prerelease 2
...
in this case: 0.0.1< 0.0.1.1 pre 1< 0.0.1.1< 0.0.2 pre1< 0.0.2 pre2
< ...< 0.0.2
It should also be possible to skip 0.0.2 (people actually do such
things ;-))
libjson 0.0.1 is released (a final release)
libjson 0.0.2 alpha1 is released --> prerelease 1
libjson 0.0.2 alpha2 is released --> prerelease 2
libjson 0.0.2 beta1 is released --> prerelease 3
libjson 0.0.2 beta2 is released --> prerelease 4
libjson 0.0.2 rc1 is released --> prerelease 5
libjson 0.0.3 rc1 is released -->prerelease 1
libjson 0.0.3 is released --> final release
so: 0.0.1< 0.0.2 pre1< 0.0.2 pre2< ...< 0.0.3 pre1< 0.0.3
a user not wanting to use prereleases would just skip all preX in the
above examples.
Ok, I think I understand so far. I was thinking something similar. But
is a four digit version really necessary?
Now regarding snapshot versions: First we have to simplify the problem:
I think we should only support a linear system, so we assume there's
only one master repository and one branch where packages are created
from.
Now we still have the problem that git/hg etc revision numbers cannot
be easily compared (is 5363aed42ff7f2edd796 more recent than
882cc02a58797a313a62 ?).
So I suppose the following: A git/hg/... snapshot always has a 'base'
release. This is the release the snapshot is based on. A snapshot is
always more recent than it's base release:
release1 0.0.1
snapshot1 5363aed42ff7f2edd796 base:0.0.1
so release 1< snapshot1
snapshots can be based on pre-relases
release1 0.0.1
pre-release1 0.0.2-pre1
snapshot1 5363aed42ff7f2edd796 base:0.0.2-pre1
pre-release2 0.0.2-pre2
so release1< pre-release1< snapshot1< pre-release2
A git snapshot always only replaces it's base release! If there's a
newer base release, the git snapshot is considered to be 'old'.
No how do we sort multiple snapshots based on the same base release?
I think a date based approach makes sense(with second or minute
resolution?)
So we now have this final (and complex) example:
libjson 0.0.1 is released (a final release) #1
libjson snapshot 2011.06.19.18.45 base:0.0.1 #2
libjson snapshot 2011.06.19.18.46 base:0.0.1 #3
libjson 0.0.2 alpha1 is released --> prerelease 1 #4
libjson snapshot 2011.06.19.18.45 base:0.0.2-pre1 #5
libjson snapshot 2011.06.19.18.50 base:0.0.2-pre1 #6
libjson 0.0.2 alpha2 is released --> prerelease 2 #7
libjson 0.0.1.1 rc1 is released --> prerelease 1 #8
libjson snapshot 2011.06.19.18.45 base:0.0.1.1-pre1 #9
libjson snapshot 2011.06.19.18.50 base:0.0.1.1-pre1 #10
libjson 0.0.1.1 is released (a final release) #11
libjson 0.0.2 beta1 is released --> prerelease 3 #12
libjson 0.0.2 beta2 is released --> prerelease 4 #13
libjson 0.0.2 rc1 is released --> prerelease 5 #14
libjson 0.0.3 rc1 is released -->prerelease 1 #15
libjson 0.0.3 is released --> final release #16
0.0.1< snapshot #2< snapshot #3< 0.0.1.1-pre1< snapshot #9<
snapshot #10< 0.0.1.1< 0.0.2-pre1< snapshot #5< snapshot #6<
0.0.2-pre2< 0.0.2-pre3< 0.0.2-pre4< 0.0.2-pre5< 0.0.3-pre1< 0.0.3
Someone with snapshots enabled will get updates like presented in the
above chain, with one exception: "0.0.2-pre1" is released
immediately after "snapshot #3" so the releases 'between' "snapshot #3"
and "0.0.2-pre1" will be skipped!
With snapshots disabled the update path looks like this:
0.0.1< 0.0.1.1< 0.0.3
This got quite complex. When I was thinking about SCM integration I was
thinking about you only specify the address to the repository, which
will mean the latest commit on the main branch. Then you could also
specify tags, branches and perhaps specific commits. But you could never
specify, for example, a release (or commit) newer then another commit.
This wouldn't work:
orb "dwt", "~> 0.3.4", :git => "git://github.com/jacob-carlborg/libjson.git"
I see now that I've specified a version in the git example on the wiki.
This was a mistake, I removed the version now.
--
/Jacob Carlborg
