On 2011-07-15 09:14, Nick Sabalausky wrote:
"Jacob Carlborg"<d...@me.com> wrote in message
news:ivkrdj$ci4$1...@digitalmars.com...
I've written a more formal specification of my ideas for a package manager
for D.
https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D
Note that I am exploring the possibility of using D as the language for
all the files mentioned in the link above.
The current status is that building packages and installing them works,
but quite limited. No dependency tracking or central repository so far.
Please comment and suggest.
Good start :) Here are my random thoughts on it (sorry if you've already
answered some of them, I haven't read the rest of this thread yet):
- I find the "orb" vs "orbit" distinction a little confusing and
unnecessary. Why not just call it all "orb"? (Or call everything "orbit"?)
This is how I'm thinking: Orbit is the name of the package manager,
"orb" is what you type on the command line when interacting with the
package manager. A package is also called an "orb".
- I think the files "orbfile", "{name}.orbspec" and "metadata" (the one
inside the packages) should all just be one file. Frankly, the existence of
all three of them confuses the hell out of me.
Ok, this is how it works:
"orbfile" is a completely optional file listing all packages a project
depends on, it shouldn't contain anything else. It allows you to run
"orb install" in the project directory to install all needed packages.
Just because a project uses packages doesn't mean it self needs to be a
package. The orbspec file requires more than just listing dependencies.
I guess the tool could look for an .orbspec file and install all its
dependencies. But what happends if it finds several .orbspec files in a
directory. This is just like the Gemfile for those how have used
bundler: http://gembundler.com/
"orbspec" is a specification of how a package looks like and what it
contains. It's intended for creating a package out of a project.
"metadata" is basically the orbspec copied into the archive. I was first
thinking about "compiling" down the Ruby code into YAML or JSON but for
now the Ruby code is included in the archive.
- Are the subtypes a "pick one" or "pick all that are included" deal? I
think that latter would make more sense.
I'm thinking this can be inferred from other fields like "executables"
and "libraries". A package could contain several types.
- Instead of "~> 0.3.4", what about "> 0.3+.4+"? Or ""~> 0.3+.4+"? Or
something vaguely like that. That would be more flexible.
So you mean I can have a version like this: "> 0.3.4+" meaning any
version from "0.3.4" to "0.3.9"? It might be a good idea.
- What happens if someone tries to upload a newer version with the same old
version number? Or if they forcefully do it?
I haven't thought about that. It probably shouldn't be possible.
- Are version numbers allowed to have more or less than three parts? I think
they should. Do version comparisons still work on version numbers with an
arbitrary number of parts? Again, I think they should.
I was hoping to only have version numbers with three parts. If fewer
parts are used it would probably easiest to infer a 0 for the missing
parts, i.e. "1" == "1.0.0".
Is there a need for more parts than three? The whole idea of having
three version parts is to be able to use "~> 0.3.4". But if "> 0.3.4+"
would be allowed then arbitrary number of parts could be allowed.
- Is version "2.10" after "2.1" or are they the same? What about "2.01" vs
"2.1"? I would vote for "2.10> 2.1 == 2.01", because I see version "parts"
as distinct numbers separated by a period, rather than fractional numbers.
I haven't thought about that. I see version parts as distinct numbers as
well.
- It should allow boolean operators and parens for the version selections.
For instance: "(>= 2.1&& <= 2.6&& != 2.4) ||>= 3.4" (Ie, "Any version
from 2.1 through 2.6, but 2.4 has critical bugs, and 3.4+ contains a 2.x
compatibility layer.")
Hehe. Now this is getting quite complicate, but it would be nice to have
yes. Not something I will aim for in the first release.
- There should be a "list" command and a "list {package name}" command to
see what's installed. Maybe even "list {package name} {version expression}"?
And maybe something too see what's available but not yet installed? They
should all list the versions in guaranteed-sorted order so you can see which
are the newest and oldest installed versions by looking at the first and
last.
Absolutely, I've completely forgot about the "list" command.
- At some point, 7z should be supported (and tarballs, of course).
If someone is willing to create D module of create bindings for any
available libraries. Any libraries the forces a specific license is out
of the question.
- How should platforms be handled WRT packages? Ie, Do all platforms need to
be in the same orb package? Do they all need to be in separate packages?
Either way? If they're not all required to be in the same package, how does
orb find the package that had the right platform?
I good questions. I haven't given binary packages that much thought. I
was first going for a source only package manager that requires all
packages to be built before installed.
I see three options:
* One package for all platforms
* Include the platform in the package name and in the orbspec
* Have a sub path (on the server) for every platform, i.e.:
dorbit.org/orbs/linux/dwt-1.3.2.orb.zip
- Is it really necessary to have separate "build_dependencies" and
"runtime_dependencies"? And why have both "runtime_dependencies" and "orbs"
instead of just picking one name and sticking with it?
There is no runtime dependency on something that is statically linked.
Therefore it would be unnecessary to do a permanent installation on
those dependencies. The user could get an option to either permanently
installed these dependencies or to temporarily install them.
The other way around would be possible as well. A package can depend on
a dynamically linked library and use it only through function pointers.
Then the package would only have a runtime dependency on the library,
i.e. it wouldn't be needed when building.
- Would it be a good idea to have and additional field "extra" for
non-standard expandability? So people could add extra fields they felt would
be useful, like "extra.foo" and "extra.bar", etc. And popular ones could
eventually be formally added to the specification as just simply "foo" and
"bar".
I guess there could be an "extra" field accepting a hash. How would the
field be used, by other tools?
- What's the point of the fields "files", "libraries" and "executables"?
Seems like extra work for no real benefit.
"files" is basically all files that should be put in the package.
"libraries" and "executables" are all libraries and executables that
should be installed (regardless if they are pre-compiled or built during
installation).
- This supports having multiple versions of the same package installed at
the same time, right? If not it should.
Yes, that's the whole point of having version, as I see it.
- I see there's an "upgrade" callback, but I didn't see an "upgrade"
command. Is upgrading in or out? I think that there should be an "upgrade"
command that upgrades the installed versions of packages as far as it can
*without* breaking any other installed packages that depend on it. Ex, if
Foo requires Bar v2.6 or earlier, and SuperFoo requires Bar v2.7 or earlier,
and Bar v2.3 is installed, but the latest Bar is v2.9, then "upgrade bar"
would upgrade Bar v2.3 to v2.6 and display a message that says "Bar upgraded
from 2.3 to 2.6, but the newest is 2.9, run "orb install Bar" to install the
newest Bar, too." (Or maybe it should install both 2.6 and 2.7? Or one/both
of those and 2.9?) For upgrading, we should also think about how to do
upgrades without clobbering any of it's settings.
I guess I didn't think this through. I think it will require some thought.
- For POSTing a package to a repository, how does authentication work? All
repos don't have to provide unrestricted upload access do they?
I haven't thought about this more than there will be some kind of
authentication. Probably HTTP basic authentication.
- I'm not sure I understand how the "source" command works. Can it be
provided more than once? And then it just picks the first one that actually
has the package?
The "source" command specifies a path to a repository where to fetch
packages from. I haven't thought about if the it can be provided more
than once. It might be a good idea.
- The "central repositories" don't necessarily sound all that central, so
they probably should just be called "repositories".
Ok.
- What about default repositories? It should support that. (Kinda makes
sense, otherwise how would "orb install xxx" know where to look?) And there
should be simple commands to add/update/remove/list (and reorder?) the
default repositories. If a package A specifies a dependency B and a
repository for that dependency B, then which one has priority for
downloading B: The default repositories or the repository specified by
package A?
Yes there will be a default repository. As someone else suggested there
could be an "orbfile" in the users home directory that can contain
default settings like for "source". Or a more general config file for orbit.
- Here's a problem with using an actual programming language for the
orbfile/name.orbspec/metadata file: Suppose Orb version X uses Ruby version
Y. Then, Orb X+1 comes out and which has Ruby upgraded to Y+1. Now, someone
creates PackageA with an orbfile/name.orbspec/whatever that relies on Ruby
Y+1. Someone else still has Orb version X and tries to get PackageA. Kaboom!
Yeah, that is a problem. But I wonder how much this will be a problem in
practice. I don't think this will so big problem in practice using Ruby
as the language. On the other hand, using D, will be a big problem. D
breaks something in every release.
Therefore, the orbfile/name.orbspec/whatever needs to specify which version
of Orb (or Ruby) it requires. But now we have a chicken-and-the-egg problem:
How can Orb X figure out that PackageA requires Orb X+1 if Orb X can't
properly read PackageA's orbfile/name.orbspec/whatever?
That is a problem. If the metadata is "compiled" YAML/JSON then we can
get around this.
- If I install D library "libfoo", then I should be able write myapp.d with
"include foo.blah;" and then do "dmd myapp.d" *without* manually
specifying -Ipath_to_libfoo. It should just work. How will Orb handle that?
It can't. The solution to this is a build tool, as I see it. The build
tool knows about the package manager and let you specify dependencies on
package. Think about Drake, it could look like this:
target("myapp.d", {
orb("libfoo"); // automatically links with "libfoo" and includes
its header path.
});
-Ipath_to_libfoo needs somehow be passed to the compiler, and linking
with the library as well. Maybe it would be possible to manipulate the
dmd.conf/sc.ini but this seems very complicated.
And how will that interact with DVM? Ie, if I do "dvm use 2.051", then "orb
install libfoo", then "dvm use 2.054", then I should still have access to
libfoo without needing to specify -Ipath_to_libfoo.
If you do "dvm use 2.051", then "orb install libfoo" then "libfoo" will
only be installed for dmd 2.051, that's the whole point. You would have
to run "orb install libfoo" again after switching compiler. Orbit could
of course share the same package if possible.
- Where does everything get installed?
For now, on Posix, in /usr/local/orbit. If used through DVM it will be
installed in somewhere in ~/.dvm.
- In many ways this sounds a lot like a generalized DVM. Maybe Orb should
eventually take over DVM's duties by making a DMD orb package.
No, I don't think so. DVM is quite specialized in what it does.
Manipulating the PATH variable (or the registy on Windows) to be able to
do what it does. I don't what to mix DVM and Orbit.
--
/Jacob Carlborg
