Re: dub: should we make it the de jure package manager for D?
On Friday, 27 September 2013 at 06:49:39 UTC, Jacob Carlborg wrote: On 2013-09-26 13:39, Dicebot wrote: No, you just let maintainers interested in target systems to take care of it. And package for 2-3 you care about _personally_. It is an obsolete idea that developer of a library/program should do any packaging at all. If your program is any good, there will always be volunteers to adapt it nicely for their beloved systems. How about doing a compromise. We add a new command to dub which will execute an installed package. Something like this: $ dub install foo $ dub exec foo --help This will by default look in the package.json file, in the current working directory, if it exists, for the correct version of the package to run. I like the idea of a dub exec for when there is a tool you want to make use of during development. Keeps everything nice and tidy and doesn't pretend to be an OS package manager. +1
Re: dub: should we make it the de jure package manager for D?
On 08/10/2013 13:38, Jacob Carlborg wrote: On 2013-10-08 12:51, Bruno Medeiros wrote: From what I understand, for dependency graph locking to work at all, then each package (as stored in the central package repository) would have to specify its full dependency graph in the package specification. So the foo package would have to specify not only the bar dependency, but also xyz=0.0.1 as a dependency. Isn't that how it would work? No, now necessarily. Using Bundler it works like this: * You have your Gemfile, corresponds to package.json in dub * You run bundle install * When you do that it will create a new file, Gemfile.lock. This file contains the complete dependency graph of what it just installed. * Running bundle install when Gemfile.lock exist, it will read Gemfile.lock instead of Gemfile Rigth, but then you would need to share the .lock file to the other machine you want to install/compile foo in. The repository alone would not be enough for that. Rather, I think dub should adopt Semantic Versioning as part of its recommended practices for package versioning: http://semver.org/spec/v2.0.0.html In this practice, stuff like xyz: =0.0.1 is not recommended, an upper bound on the version is required, to allow breaking changes in xyz. Semantic versioning helps, but it won't solve the problem. I wouldn't trust that a library actually follows the semantic versioning scheme. It's quite easy to accidentally add new API without incrementing the middle number. Or break the API. Hum that is true. Especially without a tool that can verify API compatibility. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 2013-10-10 14:25, Bruno Medeiros wrote: Rigth, but then you would need to share the .lock file to the other machine you want to install/compile foo in. The repository alone would not be enough for that. Yes, absolutely. In the Rails world it's recommended that you add both Gemfile and Gemfile.lock to the SCM for you're Rails applications. For Rails plugins you should only add Gemfile to SCM. You usually don't have your Rails application uploaded to RubyGems. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 27/09/2013 14:00, Wyatt wrote: On Thursday, 26 September 2013 at 14:20:53 UTC, Bruno Medeiros wrote: To be fair, I don't entirely agree with this. End user distribution/installation doesn't have to be system-specific. I can see many interesting and useful use-cases for a cross-platform user application package manager. It might be a real cool project if done properly. That's CPAN. You just described CPAN. It supports both system and user level package installing, bails properly when build deps are missing, and has enough metadata that we can support it from the system package manager for proper system-level depgraph and file tracking. If there's a wheel we want to steal or reinvent, this is THE one. But I think it's important to remember it would be complete cat puke like ruby packaging if they didn't have PAUSE [0]; clear, moderately-strict submission guidelines [1] (especially for newcomers); and PrePAN, for feedback and discussion BEFORE it goes up. The latter, and community involvement in general, is possibly the most important aspect of this process because the community is ultimately your userbase. (It's not so knock-down/drag-out as Phobos module reviews, but it's a great source of sanity checking.[2]) -Wyatt [0] http://www.cpan.org/modules/04pause.html Recommend reading the whole thing. [1] http://search.cpan.org/~rjbs/perl-5.18.1/pod/perlnewmod.pod [2] http://prepan.org/module/nXWJ8Y9sBtw A good recent example I would only consider a cross-platform user application package manager to be interesting (that is, useful to a significant level) if it's not tied to any particular language (or platform). -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 26/09/2013 21:05, Jacob Carlborg wrote:
Are... you... serious?... O_O
There are incredibly important benefits for development-time usage.
To automatically fetch the required dependencies, making sure they are
correct for you application. Easily upgrade the version of dependencies?
Support multiple build configurations (including for example unittest
runners)?
Supporting all the previous functionality in a way that is
replicable across different machines?
It's not, that's another big issue with dub. It doesn't lock the
dependency graph. If you have the packages:
My own package:
{
name: foo,
dependencies: {
bar: =0.0.1
}
}
Third party package:
{
name: bar,
dependencies: {
xyz: =0.0.1
}
}
Another third party package:
{
name: xyz
}
Say I install package foo on a machine. It will install package
bar-0.0.1 and xyz-0.0.1 just as it should. Then an hour later I
install the same package, foo, on a different machine. Then suddenly
it install xyz-0.0.2. Surprise surprise, it's not the same application
anymore and you have no idea if the application is working or not.
Of course, it's unlikely this will happen within an hour. But try six
months later and you might end up with a completely different application.
From what I understand, for dependency graph locking to work at all,
then each package (as stored in the central package repository) would
have to specify its full dependency graph in the package specification.
So the foo package would have to specify not only the bar dependency,
but also xyz=0.0.1 as a dependency. Isn't that how it would work?
If so, I think that might be too constricting, and might introducing
other kinds of problems and limitations. (what if you know xyz=0.0.2 is
safe to use, and want to install foo with that?)
Rather, I think dub should adopt Semantic Versioning as part of its
recommended practices for package versioning:
http://semver.org/spec/v2.0.0.html
In this practice, stuff like xyz: =0.0.1 is not recommended, an
upper bound on the version is required, to allow breaking changes in xyz.
--
Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 08/10/2013 11:51, Bruno Medeiros wrote: Rather, I think dub should adopt Semantic Versioning as part of its recommended practices for package versioning: http://semver.org/spec/v2.0.0.html In this practice, stuff like xyz: =0.0.1 is not recommended, an upper bound on the version is required, to allow breaking changes in xyz. Doh, I should have checked this before I posted, but dub does this already: http://code.dlang.org/package-format#version-specs That means all packages (at least production ready ones) should use upper bounds on their dependencies. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 27/09/2013 18:12, Sönke Ludwig wrote: Am 27.09.2013 14:04, schrieb Jacob Carlborg: On 2013-09-10 22:48, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? The name package.json is already used by NPM (node package manager). Should we rename the package file or should we just ignore the conflict? Although the conflict will probably not matter in practice, I think just dub.json would be better. The name package.json is still from the early days when it was called VPM. Any chance that Dub packages could be called something else instead, such as bundles, throughout the documentation, wiki, etc.? The point here would be to limit cognitive confusion/overlap with D language packages. It's just a minor cleanup I know, and of subjective value, but still I think it would be nicer... -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On Tuesday, 8 October 2013 at 10:18:03 UTC, Bruno Medeiros wrote: I would only consider a cross-platform user application package manager to be interesting (that is, useful to a significant level) if it's not tied to any particular language (or platform). ...Oh, you're looking for Gentoo Prefix. Got it. Here you go: http://www.gentoo.org/proj/en/gentoo-alt/prefix/ :) -Wyatt
Re: dub: should we make it the de jure package manager for D?
On 2013-10-08 12:51, Bruno Medeiros wrote: From what I understand, for dependency graph locking to work at all, then each package (as stored in the central package repository) would have to specify its full dependency graph in the package specification. So the foo package would have to specify not only the bar dependency, but also xyz=0.0.1 as a dependency. Isn't that how it would work? No, now necessarily. Using Bundler it works like this: * You have your Gemfile, corresponds to package.json in dub * You run bundle install * When you do that it will create a new file, Gemfile.lock. This file contains the complete dependency graph of what it just installed. * Running bundle install when Gemfile.lock exist, it will read Gemfile.lock instead of Gemfile By the way, the registry (package repository) need to be able to figure you the complete dependency graph, otherwise it won't be very effective. It should only require one single HTTP request to the registry to get the complete list of dependencies of a given package. If so, I think that might be too constricting, and might introducing other kinds of problems and limitations. (what if you know xyz=0.0.2 is safe to use, and want to install foo with that?) The way I've done that using Bundler is to explicitly specify that as a direct dependency, in this case of foo. I don't know if there's a better way. Rather, I think dub should adopt Semantic Versioning as part of its recommended practices for package versioning: http://semver.org/spec/v2.0.0.html In this practice, stuff like xyz: =0.0.1 is not recommended, an upper bound on the version is required, to allow breaking changes in xyz. Semantic versioning helps, but it won't solve the problem. I wouldn't trust that a library actually follows the semantic versioning scheme. It's quite easy to accidentally add new API without incrementing the middle number. Or break the API. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-10-08 12:55, Bruno Medeiros wrote: Any chance that Dub packages could be called something else instead, such as bundles, throughout the documentation, wiki, etc.? The point here would be to limit cognitive confusion/overlap with D language packages. It's just a minor cleanup I know, and of subjective value, but still I think it would be nicer... What if it always said dub package? There's always the chance of confusion. On Mac OS X a bundle is a special folder containing an executable or dynamic library plus its resources. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 27/09/2013 12:53, Jacob Carlborg wrote: On 2013-09-27 13:22, Bruno Medeiros wrote: It compiles packages when used as dependencies in another package, and run with the dub build command. What perhaps you meant to say is that dub doesn't compile application packages that it downloads from the registry. That might be true. Even so, if you do dub install of one such package, then navigate to the directory where it was installed, and do dub build, it should compile the executable. I haven't tried that myself though. Yes, it does. But that's where I draw the line with the dub install command actually installs or not. So since it doesn't build the package I wouldn't considered the package installed, which is what I expect from a command named install. But what you already have said cache sounds like a better name for what it currently does. mirror would also be a good name for what it currently does. But personally I'm not that bothered about install. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 2013-09-27 02:20, Jonathan M Davis wrote: This is definitely something that needs to be sorted out. I talked with Sönke about this in the dub forum. He thinks the current behavior should be default and could possibly add a new flag, --lock-dependencies, which locks the dependency graph. I think it should be the other way around. The dependency graph should be locked by default. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-26 13:39, Dicebot wrote: No, you just let maintainers interested in target systems to take care of it. And package for 2-3 you care about _personally_. It is an obsolete idea that developer of a library/program should do any packaging at all. If your program is any good, there will always be volunteers to adapt it nicely for their beloved systems. How about doing a compromise. We add a new command to dub which will execute an installed package. Something like this: $ dub install foo $ dub exec foo --help This will by default look in the package.json file, in the current working directory, if it exists, for the correct version of the package to run. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Friday, 27 September 2013 at 06:49:39 UTC, Jacob Carlborg wrote: On 2013-09-26 13:39, Dicebot wrote: No, you just let maintainers interested in target systems to take care of it. And package for 2-3 you care about _personally_. It is an obsolete idea that developer of a library/program should do any packaging at all. If your program is any good, there will always be volunteers to adapt it nicely for their beloved systems. How about doing a compromise. We add a new command to dub which will execute an installed package. Something like this: $ dub install foo $ dub exec foo --help This will by default look in the package.json file, in the current working directory, if it exists, for the correct version of the package to run. Ok, this is pretty hygienic (though as I have said it makes more sense to call it `dub cache` instead of `dub install`). Though what does it give you over just providing same environment via build dependencies? (I know, dub does not seem to build binaries from dependencies right now but I got an impression this is going to be fixed)
Re: dub: should we make it the de jure package manager for D?
On 2013-09-27 09:08, Dicebot wrote: Ok, this is pretty hygienic (though as I have said it makes more sense to call it `dub cache` instead of `dub install`). Currently cache is probably a better name. But if binaries are compiled I think install is an ok name. It just doesn't install it in the usual locations. Though what does it give you over just providing same environment via build dependencies? I'm not sure what you mean. (I know, dub does not seem to build binaries from dependencies right now but I got an impression this is going to be fixed) Again, I'm not sure what you mean by from dependencies. It doesn't build binaries at all. Preferably I would like to be able to use dub install/cache and dub exec regardless if I have a project/package.json or not. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Friday, 27 September 2013 at 07:32:12 UTC, Jacob Carlborg wrote: On 2013-09-27 09:08, Dicebot wrote: Ok, this is pretty hygienic (though as I have said it makes more sense to call it `dub cache` instead of `dub install`). Currently cache is probably a better name. But if binaries are compiled I think install is an ok name. It just doesn't install it in the usual locations. It won't install it out of clone dir either in that mode. Why would it? Building is enough. Though what does it give you over just providing same environment via build dependencies? I'm not sure what you mean. Currently you can define dependencies in your package.json to other dub packages. Those will be in your -I flags when building. I expect this to be also extended to -L and PATH, so that you can call any binaries from dependency packages as if they were installed (during build/test of your package). For this to work no real installation is needed, just building packages straight in the clone dir and keeping it.
Re: dub: should we make it the de jure package manager for D?
On 26/09/2013 21:11, Jacob Carlborg wrote: On 2013-09-25 18:14, Bruno Medeiros wrote: But this is all for development-time usage. To have the same tool try to be an executable installation manager is another thing entirely and, in my opinion quite ill-suited for dub (see related OP). Where did this idea even come from?? We can take Eclipse as an other example. It has a built in package manager for plugins. Would you prefer that it just downloaded Java source files. Then you have to manually build the plugin and figure out where to place it to have Eclipse recognize it? No, I wouldn't prefer that. I'm not saying it's not useful to have an end-user package manager. It is. But I think that the development use-cases and functionality are more important, and therefore dub should focus primarily (if not exclusively) on those. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 26/09/2013 20:52, Jacob Carlborg wrote: You have to install the yourself, yes. Not compile it. Dub should take care of the compiling aspect. Have you tried it? It does _not_ compile a package when running dub install. It compiles packages when used as dependencies in another package, and run with the dub build command. What perhaps you meant to say is that dub doesn't compile application packages that it downloads from the registry. That might be true. Even so, if you do dub install of one such package, then navigate to the directory where it was installed, and do dub build, it should compile the executable. I haven't tried that myself though. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 2013-09-27 13:22, Bruno Medeiros wrote: It compiles packages when used as dependencies in another package, and run with the dub build command. What perhaps you meant to say is that dub doesn't compile application packages that it downloads from the registry. That might be true. Even so, if you do dub install of one such package, then navigate to the directory where it was installed, and do dub build, it should compile the executable. I haven't tried that myself though. Yes, it does. But that's where I draw the line with the dub install command actually installs or not. So since it doesn't build the package I wouldn't considered the package installed, which is what I expect from a command named install. But what you already have said cache sounds like a better name for what it currently does. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-27 10:05, Dicebot wrote: It won't install it out of clone dir either in that mode. Why would it? Building is enough. I'm not going to argue the semantics of install. But yes, building is enough for that command. Currently you can define dependencies in your package.json to other dub packages. Those will be in your -I flags when building. I expect this to be also extended to -L and PATH, so that you can call any binaries from dependency packages as if they were installed (during build/test of your package). For this to work no real installation is needed, just building packages straight in the clone dir and keeping it. If that's supported then dub exec won't give any advantage. But since that's currently isn't supported I suggested the dub exec command. Unless dub exec can be used without a package.json file. That would work with packages you previously run dub install for. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Friday, 27 September 2013 at 11:50:28 UTC, Jacob Carlborg wrote: If that's supported then dub exec won't give any advantage. But since that's currently isn't supported I suggested the dub exec command. I am pretty sure everyone has agreed it should be supported in the long run (and implementation effort is roughly the same as for `dub exec`). It naturally fits with current dub concept and mode of operations. Don't exactly know what are Sonke plans about it though.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-10 22:48, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? The name package.json is already used by NPM (node package manager). Should we rename the package file or should we just ignore the conflict? -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Friday, 27 September 2013 at 13:00:58 UTC, Wyatt wrote: ... afaik CPAN mixes development packages and distribution packages too, which I doubt is desired. It may be unfortunate consequence of Perl interpreted nature though.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 14:20:53 UTC, Bruno Medeiros wrote: To be fair, I don't entirely agree with this. End user distribution/installation doesn't have to be system-specific. I can see many interesting and useful use-cases for a cross-platform user application package manager. It might be a real cool project if done properly. That's CPAN. You just described CPAN. It supports both system and user level package installing, bails properly when build deps are missing, and has enough metadata that we can support it from the system package manager for proper system-level depgraph and file tracking. If there's a wheel we want to steal or reinvent, this is THE one. But I think it's important to remember it would be complete cat puke like ruby packaging if they didn't have PAUSE [0]; clear, moderately-strict submission guidelines [1] (especially for newcomers); and PrePAN, for feedback and discussion BEFORE it goes up. The latter, and community involvement in general, is possibly the most important aspect of this process because the community is ultimately your userbase. (It's not so knock-down/drag-out as Phobos module reviews, but it's a great source of sanity checking.[2]) -Wyatt [0] http://www.cpan.org/modules/04pause.html Recommend reading the whole thing. [1] http://search.cpan.org/~rjbs/perl-5.18.1/pod/perlnewmod.pod [2] http://prepan.org/module/nXWJ8Y9sBtw A good recent example
Re: dub: should we make it the de jure package manager for D?
On Friday, 27 September 2013 at 13:03:20 UTC, Dicebot wrote: On Friday, 27 September 2013 at 13:00:58 UTC, Wyatt wrote: ... afaik CPAN mixes development packages and distribution packages too, which I doubt is desired. It may be unfortunate consequence of Perl interpreted nature though. Unless we're talking about different things, that's denoted in the META.json (I don't recall if the distinction existed in the META.yaml era) with e.g. release_status : stable, or release_status : testing, A proper tool can (and will) filter for this by default. And to be explicitly clear, I'm not saying it's perfect for us as-is (or even perfect in general). They're just the model citizen to take after. -Wyatt
Re: dub: should we make it the de jure package manager for D?
On Friday, 27 September 2013 at 13:29:26 UTC, Wyatt wrote: Unless we're talking about different things, that's denoted in the META.json (I don't recall if the distinction existed in the META.yaml era) with e.g. release_status : stable, or release_status : testing, No, I mean that it does not make any difference between programs written in perl to be widely used and libraries / tools for development in perl. For interpreted languages this distinction hardly makes sense, but for natively compiled ones it is pretty important. And that was kind of the core point of the argument - can language-specific package manager be allowed to manage the former? I stand by the point that is a bad practice to provide any functionality that allows it, Jacob has opposite point of view :)
Re: dub: should we make it the de jure package manager for D?
Funny, I started this with a clear mindset, but it ended up more nuanced than I expected. On Friday, 27 September 2013 at 13:43:59 UTC, Dicebot wrote: No, I mean that it does not make any difference between programs written in perl to be widely used and libraries / tools for development in perl. For interpreted languages this distinction hardly makes sense, but for natively compiled ones it is pretty important. That's true, though at some level I wonder if its continued prevalence more comes back to CPAN supporting system and local-to-user installation simultaneously. Users have been adding ~/bin to their $PATH since $EPOCH. And that was kind of the core point of the argument - can language-specific package manager be allowed to manage the former? Not at the system level, it probably shouldn't. Ideally, binary executables and libraries are the exclusive demesne of the system package manager. As a historical footnote, I suspect part of why CPAN works the way it does is it was designed and (I think) released before RPM was even written. (It may even predate the original dpkg.) More important than long precedent, though: Perl is largely self-contained by default, and can be made completely so [0]. In some sense, I _like_ the idea of installing module source to e.g. /usr/lib64/d2/site_d/2.63/ as long as dub is built into a normal install (control env with a config file in ~) and let it update the user's dmd.conf as a value-add [1]. I stand by the point that is a bad practice to provide any functionality that allows it, Jacob has opposite point of view :) I... turns out I have mixed feelings on this. In the general sense, I'm strongly inclined to agree with you. External package management undermines the sysadmin's ability to make good package installation decisions, splits the system dependency graph, and can create file conflicts. But on the other hand, as long as there are sufficient options (preferable), a utility, or an API to interrogate dub for complete useful metadata; I'll refrain from getting too vehement about it. This is only because that means system package managers _can_ properly integrate it, should they choose to do so. (cf. g-cpan [2]) See, I have my standards as a sysadmin, but don't forget policy victims, either: many others are stuck dealing with a CErtaiN Terrible Old diStribution and would probably appreciate a tool that isn't stuck mired in Really HorriblE poLicy when they're trying to get work done. [3] My breaking point is actually the idea of using dub to call the system package manager, which I still think is a Really Bad Idea. [4] -Wyatt [0] Consider the output of perl '-V:.*site.*' and the existence of Perlbrew (http://perlbrew.pl/) [1] As a tangent, a nice separate utility to have might be something like a dub-config to help with module paths and maybe linkage (In the vein of all those /usr/bin/*-config things). Like so: dmd `dub-config --moddir vibed` thisLinksVibe.d. [2] http://www.gentoo.org/proj/en/perl/g-cpan.xml [3] Yes, juvenile RHELL potshots make me feel a little better about my situation. [4] https://github.com/rejectedsoftware/dub/issues/52
Re: dub: should we make it the de jure package manager for D?
Am 27.09.2013 14:04, schrieb Jacob Carlborg: On 2013-09-10 22:48, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? The name package.json is already used by NPM (node package manager). Should we rename the package file or should we just ignore the conflict? Although the conflict will probably not matter in practice, I think just dub.json would be better. The name package.json is still from the early days when it was called VPM.
Re: dub: should we make it the de jure package manager for D?
On Tuesday, 10 September 2013 at 20:48:58 UTC, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Andrei Well, you've started the mother of all bikeshed threads, as every dev/sysadmin has a different opinion on how to handle packages. ;) There have been some valid concerns raised about whether dub should do all the stuff it does, but I think they're largely moot at this point, given the small size of the D community. Better a tool that does everything initially, as current D users are inevitably developers, which can then be split up and specialized later if necessary. I've used dub on FreeBSD and was impressed with both the tool and how clean the code is, even though there were small linking/library incompatibilities that meant I had to hack a workaround into dub to compile vibe.d. Sonke was remarkably responsive to my feedback, I cannot imagine there will be a better option for a D package manager and developer.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-25 18:14, Bruno Medeiros wrote: But this is all for development-time usage. To have the same tool try to be an executable installation manager is another thing entirely and, in my opinion quite ill-suited for dub (see related OP). Where did this idea even come from?? If dub doesn't install packages, why the h*ll should I use it in the first place? I can just use the system package manager. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-25 17:51, Bruno Medeiros wrote: Whoa, no. Application/executable install management as a goal would be a ridiculously bad idea. Because that would sit at the wrong abstraction level. The OS package manager should not be tied to a particular language to compile packages from. Does it makes any sense to have to use D's package manager if my cmd-line util is written in D, but if I have a C++ or Go derived executable, I would have to use a different package manager for each? And what if I want my tool to depend (at runtime) on an executable generated from another language? Devise a mechanism for cross-package-manager interoperaction?... Ridiculous. An application/executable manager should be language agnostic (and not even require compilation). Instead I need to package my application and libraries for all the various of package managers out there. Not to mention neither Mac OS X or Windows comes with a package manager installed by default. It's like buying a car. I buy a car for getting from A to B. I have bought my car and prepares to get from A to B. The car won't start, hmm ..., oh it has no engine. I have to figure out myself how to buy and install the engine. It's only half way there. It's the same with dub. I install a package to use the tool. But wait, it actually _don't_, it just clones the repository. I have to figure out myself how to compile and install the tool. It's only half way there. What dub should be first and foremost is a structured build tool (and build specification) for D projects. There's nothing wrong with being a build tool. But currently dub tries to be way more than a build tool. I don't think a build tool should have any business in downloading packages, or download anything. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 11 September 2013 06:48, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Is there a front-end for dub floating around? Something like the cygwin package installer comes to mind... (nothing fancy) I've used dub once, I was very satisfied with the experience. I think a front-end where you can browse the repository and select/deselect packages conveniently would be a great addition to the experience, if it's not already available.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-26 09:44, Manu wrote: Is there a front-end for dub floating around? Something like the cygwin package installer comes to mind... (nothing fancy) I've used dub once, I was very satisfied with the experience. I think a front-end where you can browse the repository and select/deselect packages conveniently would be a great addition to the experience, if it's not already available. I'm wondering how that would look like. Because apparently people don't like dub to actually install packages. If dub doesn't install packages I don't think it would gain much of over browsing the registry: http://code.dlang.org/ -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 26 September 2013 18:24, Jacob Carlborg d...@me.com wrote: On 2013-09-26 09:44, Manu wrote: Is there a front-end for dub floating around? Something like the cygwin package installer comes to mind... (nothing fancy) I've used dub once, I was very satisfied with the experience. I think a front-end where you can browse the repository and select/deselect packages conveniently would be a great addition to the experience, if it's not already available. I'm wondering how that would look like. Because apparently people don't like dub to actually install packages. If dub doesn't install packages I don't think it would gain much of over browsing the registry: http://code.dlang.org/ Is that not a package list? I only just skimmed this thread, but it seemed dub installs packages to me... What did it install when I told it to install vibe.d and some other companion pieces then?
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 06:59:05 UTC, Jacob Carlborg wrote: On 2013-09-25 18:14, Bruno Medeiros wrote: But this is all for development-time usage. To have the same tool try to be an executable installation manager is another thing entirely and, in my opinion quite ill-suited for dub (see related OP). Where did this idea even come from?? If dub doesn't install packages, why the h*ll should I use it in the first place? I can just use the system package manager. Install packages: yes, but locally, primarily for dub/development use. Not system-level library installations for end-users. I think when people are talking about not wanting it to install things they are talking about the latter, not the former.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 07:44:25 UTC, Manu wrote: I've used dub once, I was very satisfied with the experience. I think a front-end where you can browse the repository and select/deselect packages conveniently would be a great addition to the experience, if it's not already available. Do you mean a tool that edits package.json file? I see little gain in having GUI for that.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 06:59:05 UTC, Jacob Carlborg wrote: On 2013-09-25 18:14, Bruno Medeiros wrote: But this is all for development-time usage. To have the same tool try to be an executable installation manager is another thing entirely and, in my opinion quite ill-suited for dub (see related OP). Where did this idea even come from?? If dub doesn't install packages, why the h*ll should I use it in the first place? I can just use the system package manager. Good look getting all those small trivial libraries into official repositories. Actually, some distros even prohibit pure source packages at all. You should install it because you are tired of satisfying build dependencies manually and it is a bit more robust than git submodules. Pretty much all.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 11:17:29 UTC, Dicebot wrote: .. :%s/look/luck/g
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 10:18:07 UTC, John Colvin wrote: Install packages: yes, but locally, primarily for dub/development use. Not system-level library installations for end-users. I think when people are talking about not wanting it to install things they are talking about the latter, not the former. I am against even user-wide installation. It should provide environment for building other dub packages, not interfere with normal way system operates. Lot of developers are terrible at packaging and once you provide even limited tool to do it dirty way it will be forced on users. This has already made ruby projects unusable for anyone but ruby developers, I don't want this for D. AFAIK currently `dub install` does hardly anything more than just cloning matching repository and remembering it in local registry. `dub cache` would have been probably better name. And `dub install --system` should really just be deprecated as very bad practice.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 06:57:34 UTC, Jacob Carlborg wrote: Instead I need to package my application and libraries for all the various of package managers out there. Not to mention neither Mac OS X or Windows comes with a package manager installed by default. No, you just let maintainers interested in target systems to take care of it. And package for 2-3 you care about _personally_. It is an obsolete idea that developer of a library/program should do any packaging at all. If your program is any good, there will always be volunteers to adapt it nicely for their beloved systems. It's like buying a car. I buy a car for getting from A to B. I have bought my car and prepares to get from A to B. The car won't start, hmm ..., oh it has no engine. I have to figure out myself how to buy and install the engine. It's only half way there. Yeah and what you propose is like mandatory requirement to have a separate home for every single electronics vendor. Want to buy something from different one? You must buy new home for it, period. (Well, feels like we are actually going there in real like _) It's the same with dub. I install a package to use the tool. But wait, it actually _don't_, it just clones the repository. I have to figure out myself how to compile and install the tool. It's only half way there. This is just plain wrong. dub takes care of proper compilation of any tool that is contained in its registry, you don't need to do anything about it. But there is no possible legitimate reasons to install it. During development you can pretty much run in locally from the source dir. For end user distribution you must go system-specific way. There's nothing wrong with being a build tool. But currently dub tries to be way more than a build tool. I don't think a build tool should have any business in downloading packages, or download anything. It is neither a build tool nor package manager. It is a tool that aggregates different possible build tools backends and takes care of resolving build dependencies for them. Nothing more, nothing less.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 12:12:04 UTC, eles wrote: On Thursday, 26 September 2013 at 11:39:50 UTC, Dicebot wrote: On Thursday, 26 September 2013 at 06:57:34 UTC, Jacob Carlborg wrote: It is neither a build tool nor package manager. It is a tool While speaking about the dub, somebode with edit rights could take care of this little glitch? On the page: http://code.dlang.org/download The link in: Debian packages are available as part of Jordi Sayol's D APT repository. should be updated from: https://code.google.com/p/d-apt/wiki/APT_Repository to: http://d-apt.sourceforge.net/
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 11:39:50 UTC, Dicebot wrote: On Thursday, 26 September 2013 at 06:57:34 UTC, Jacob Carlborg wrote: It is neither a build tool nor package manager. It is a tool that aggregates different possible build tools backends and takes care of resolving build dependencies for them. Nothing more, nothing less. I think, the max it should be able to do is to output .deb or .rpm (or .tgz) files. After, it is the job the standard install mechanism on the host OS.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 10:18:07 UTC, John Colvin wrote: On Thursday, 26 September 2013 at 06:59:05 UTC, Jacob Carlborg wrote: On 2013-09-25 18:14, Bruno Medeiros wrote: Install packages: yes, but locally, primarily for dub/development use. Not system-level library installations for end-users. I think when people are talking about not wanting it to install things they are talking about the latter, not the former. I feel that, in order for this behaviour to be truly local to the compiler and OS-independent, that the interface between the compiler and dub is required, to inform the former about the existence of these packages. Currently, this is a compiler-by-compiler approach but, if standardized, it will kinda go into language/compiler specs.
Re: dub: should we make it the de jure package manager for D?
On 26/09/2013 07:57, Jacob Carlborg wrote: On 2013-09-25 17:51, Bruno Medeiros wrote: I have to figure out myself how to compile and install the tool. It's only half way there. You have to install the yourself, yes. Not compile it. Dub should take care of the compiling aspect. If you just want to use the tool executable artifact, dub is likely not right for you. If the tool requires more complex installation, the tool developers should provide their own installer or OS distribution package. What dub should be first and foremost is a structured build tool (and build specification) for D projects. There's nothing wrong with being a build tool. But currently dub tries to be way more than a build tool. I don't think a build tool should have any business in downloading packages, or download anything. A build tool should not download anything? That is antiquated C/C++/make way of thinking. Popular build tools for modern languages all do downloading, for example Apache Maven (for Java), Gradle (for lots of different languages), and even RubyGems. You might say RubyGems is a package manager, and not a build tool. But in practice it is both actually, even if it is not called a build tool: It fullfills the the equivalent goal as structured build tools like Maven or Graddle do for other language, the distinction is only less clear becase Ruby as a language can be easily interpreted and does require an overt compilation/build phase to generate an executable. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 26/09/2013 07:59, Jacob Carlborg wrote: On 2013-09-25 18:14, Bruno Medeiros wrote: But this is all for development-time usage. To have the same tool try to be an executable installation manager is another thing entirely and, in my opinion quite ill-suited for dub (see related OP). Where did this idea even come from?? If dub doesn't install packages, why the h*ll should I use it in the first place? I can just use the system package manager. Are... you... serious?... O_O There are incredibly important benefits for development-time usage. To automatically fetch the required dependencies, making sure they are correct for you application. Easily upgrade the version of dependencies? Support multiple build configurations (including for example unittest runners)? Supporting all the previous functionality in a way that is replicable across different machines? Have other tools such as IDEs understand the source and build structure of your application/project/library?? System package managers don't support this because they are not cross-platform. Even if cross-platform wasn't an issue, there might still be several shortcomings or inadequacies with OS package managers because they are more often geared torwards end-user installation, not development-time usage. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 26/09/2013 12:39, Dicebot wrote: During development you can pretty much run in locally from the source dir. For end user distribution you must go system-specific way. To be fair, I don't entirely agree with this. End user distribution/installation doesn't have to be system-specific. I can see many interesting and useful use-cases for a cross-platform user application package manager. It might be a real cool project if done properly. I just don't think dub should aim for that goal. It should mainly be a build tool and package manager for development-time usage. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 26/09/2013 15:16, Bruno Medeiros wrote: To automatically fetch the required dependencies, making sure they are correct for you application. When I say correct here, I also mean compatible, in case that's not clear. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 14:20:53 UTC, Bruno Medeiros wrote: On 26/09/2013 12:39, Dicebot wrote: During development you can pretty much run in locally from the source dir. For end user distribution you must go system-specific way. To be fair, I don't entirely agree with this. End user distribution/installation doesn't have to be system-specific. I can see many interesting and useful use-cases for a cross-platform user application package manager. It might be a real cool project if done properly. I just don't think dub should aim for that goal. It should mainly be a build tool and package manager for development-time usage. I mean that it must respect rules and guidelines of target system. It is entirely possible to abstract lot of details away, there is a PackageKit for example it it does work pretty good. But it works by incorporating knowledge of all specific package managers and quirks inside, something I don't think we want to spend time on in dub.
Re: dub: should we make it the de jure package manager for D?
On Thursday, 26 September 2013 at 12:14:42 UTC, eles wrote: On Thursday, 26 September 2013 at 12:12:04 UTC, eles wrote: On Thursday, 26 September 2013 at 11:39:50 UTC, Dicebot wrote: On Thursday, 26 September 2013 at 06:57:34 UTC, Jacob Carlborg wrote: It is neither a build tool nor package manager. It is a tool While speaking about the dub, somebode with edit rights could take care of this little glitch? On the page: http://code.dlang.org/download The link in: Debian packages are available as part of Jordi Sayol's D APT repository. should be updated from: https://code.google.com/p/d-apt/wiki/APT_Repository to: http://d-apt.sourceforge.net/ It'll be fixed once Sönke merges this: https://github.com/rejectedsoftware/dub-registry/pull/16
Re: dub: should we make it the de jure package manager for D?
Am 26.09.2013 16:24, schrieb Bruno Medeiros: On 26/09/2013 15:16, Bruno Medeiros wrote: To automatically fetch the required dependencies, making sure they are correct for you application. When I say correct here, I also mean compatible, in case that's not clear. Even C++ has them on Windows nowadays via NuGet, http://docs.nuget.org/docs/reference/support-for-native-projects
Re: dub: should we make it the de jure package manager for D?
On 2013-09-26 13:39, Dicebot wrote: No, you just let maintainers interested in target systems to take care of it. And package for 2-3 you care about _personally_. It is an obsolete idea that developer of a library/program should do any packaging at all. If your program is any good, there will always be volunteers to adapt it nicely for their beloved systems. Most developers/packagers/people don't even know about D. They know even less about my projects. Be realistic. Yeah and what you propose is like mandatory requirement to have a separate home for every single electronics vendor. Want to buy something from different one? You must buy new home for it, period. (Well, feels like we are actually going there in real like _) I'm just being realistic. Most of these packages won't ever end up in a system package manager. It's the same with dub. I install a package to use the tool. But wait, it actually _don't_, it just clones the repository. I have to figure out myself how to compile and install the tool. It's only half way there. This is just plain wrong. dub takes care of proper compilation of any tool that is contained in its registry, you don't need to do anything about it. It does _not_ compile packages using dub install. At least not the one I created. Yes, it could compile it without any problem. It just doesn't do that when you run dub install. But there is no possible legitimate reasons to install it. During development you can pretty much run in locally from the source dir. For end user distribution you must go system-specific way. It is neither a build tool nor package manager. It is a tool that aggregates different possible build tools backends and takes care of resolving build dependencies for them. Nothing more, nothing less. It advertise itself as a package manager. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-26 15:53, Bruno Medeiros wrote: You have to install the yourself, yes. Not compile it. Dub should take care of the compiling aspect. Have you tried it? It does _not_ compile a package when running dub install. If you just want to use the tool executable artifact, dub is likely not right for you. That might be the case. I'm just seeing this as a great opportunity to use libraries and tools for development. If the tool requires more complex installation, the tool developers should provide their own installer or OS distribution package. A build tool should not download anything? That is antiquated C/C++/make way of thinking. Popular build tools for modern languages all do downloading, for example Apache Maven (for Java), Gradle (for lots of different languages), and even RubyGems. I just think that should be a separate tool. You might say RubyGems is a package manager, and not a build tool. But in practice it is both actually, even if it is not called a build tool: It fullfills the the equivalent goal as structured build tools like Maven or Graddle do for other language, the distinction is only less clear becase Ruby as a language can be easily interpreted and does require an overt compilation/build phase to generate an executable. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-26 13:17, Dicebot wrote: Good look getting all those small trivial libraries into official repositories. Actually, some distros even prohibit pure source packages at all. That's exactly why dub _should_ install packages. If you don't like, don't use it. You should install it because you are tired of satisfying build dependencies manually and it is a bit more robust than git submodules. Pretty much all. How is it more robust than git submodules? It does basically the same. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-26 16:16, Bruno Medeiros wrote:
Are... you... serious?... O_O
There are incredibly important benefits for development-time usage.
To automatically fetch the required dependencies, making sure they are
correct for you application. Easily upgrade the version of dependencies?
Support multiple build configurations (including for example unittest
runners)?
Supporting all the previous functionality in a way that is
replicable across different machines?
It's not, that's another big issue with dub. It doesn't lock the
dependency graph. If you have the packages:
My own package:
{
name: foo,
dependencies: {
bar: =0.0.1
}
}
Third party package:
{
name: bar,
dependencies: {
xyz: =0.0.1
}
}
Another third party package:
{
name: xyz
}
Say I install package foo on a machine. It will install package
bar-0.0.1 and xyz-0.0.1 just as it should. Then an hour later I
install the same package, foo, on a different machine. Then suddenly
it install xyz-0.0.2. Surprise surprise, it's not the same application
anymore and you have no idea if the application is working or not.
Of course, it's unlikely this will happen within an hour. But try six
months later and you might end up with a completely different application.
Have other tools such as IDEs understand the source and build structure of your
application/project/library??
System package managers don't support this because they are not
cross-platform. Even if cross-platform wasn't an issue, there might
still be several shortcomings or inadequacies with OS package managers
because they are more often geared torwards end-user installation, not
development-time usage.
I'm not saying that all other features are not useful but I'm saying
that it doesn't actually install the packages is a big issue.
--
/Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-26 10:39, Manu wrote: http://code.dlang.org/ Is that not a package list? I only just skimmed this thread, but it seemed dub installs packages to me... What did it install when I told it to install vibe.d and some other companion pieces then? If you run dub install it will only clone the repository and its dependencies. It will not compile anything and it won't put the resulting binaries (which don't exist) in a easily accessible location. Although it can be useful to integrate in an IDE, but that will only be useful for libraries. Not for executables, since they're not installed. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-25 18:14, Bruno Medeiros wrote: But this is all for development-time usage. To have the same tool try to be an executable installation manager is another thing entirely and, in my opinion quite ill-suited for dub (see related OP). Where did this idea even come from?? We can take Eclipse as an other example. It has a built in package manager for plugins. Would you prefer that it just downloaded Java source files. Then you have to manually build the plugin and figure out where to place it to have Eclipse recognize it? -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Thursday, September 26, 2013 22:05:35 Jacob Carlborg wrote: It's not, that's another big issue with dub. It doesn't lock the dependency graph. If you have the packages: This is definitely something that needs to be sorted out. - Jonathan M Davis
Re: dub: should we make it the de jure package manager for D?
On Thursday, September 26, 2013 15:20:53 Bruno Medeiros wrote: I just don't think dub should aim for that goal. It should mainly be a build tool and package manager for development-time usage. Agreed. I don't think that dub should have anything to with installing programs or installing anything in the system at large. IMHO, it should be solely for development. At most, it should be set up to collect all of the libraries needed for running the application so that you know what you need when actually creating a system package for your application. - Jonathan M Davis
Re: dub: should we make it the de jure package manager for D?
On 11/09/2013 12:30, Sönke Ludwig wrote: Am 11.09.2013 11:57, schrieb Jacob Carlborg: On 2013-09-10 22:48, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Unfortunately I have to say no to its current state. The biggest issue I have with dub is that it's really doesn't install packages, at least not in the traditional sense. I cannot just run dub install foo and then foo --help. It will only clone the repository, not install, or install anything. It basically only supports source packages, which makes it mostly useless for tools/application compiling to executables. I would say, compiling and installing executables is a must. It would be nice if it could compiling libraries as well. Right now it is a pure development tool. It would be very nice to have end user installs somehow supported (either by directly installing application packages or by generating OS specific packages such as DEB or RPM). But since this enters a highly operating specific area and goes into direct competition with the OS package manager, I think it needs a lot of thought and caution to be generally useful and not possibly do more harm than good in the end. But yes, it should be a primary goal in my opinion, too. Whoa, no. Application/executable install management as a goal would be a ridiculously bad idea. Because that would sit at the wrong abstraction level. The OS package manager should not be tied to a particular language to compile packages from. Does it makes any sense to have to use D's package manager if my cmd-line util is written in D, but if I have a C++ or Go derived executable, I would have to use a different package manager for each? And what if I want my tool to depend (at runtime) on an executable generated from another language? Devise a mechanism for cross-package-manager interoperaction?... Ridiculous. An application/executable manager should be language agnostic (and not even require compilation). What dub should be first and foremost is a structured build tool (and build specification) for D projects. -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On 10/09/2013 21:48, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Andrei I support making dub the official *structured build tool* for D. There seems to be a long-standing confusion and dissonance on what a package manager should do, and what would be best for D. I think what would be of the most benefit to D would be to have a structured build tool (and standard build specification). That is, there would a standard way to specify how to build D projects, supporting multiple configurations, and versioned dependencies (all done in a way such that the build tool is able to automatically fetch and manage the dependencies). These goals are well matched by dub. But this is all for development-time usage. To have the same tool try to be an executable installation manager is another thing entirely and, in my opinion quite ill-suited for dub (see related OP). Where did this idea even come from?? -- Bruno Medeiros - Software Engineer
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 25 September 2013 at 15:51:47 UTC, Bruno Medeiros wrote: Whoa, no. Application/executable install management as a goal would be a ridiculously bad idea. Because that would sit at the wrong abstraction level. The OS package manager should not be tied to a particular language to compile packages from. Does it makes any sense to have to use D's package manager if my cmd-line util is written in D, but if I have a C++ or Go derived executable, I would have to use a different package manager for each? And what if I want my tool to depend (at runtime) on an executable generated from another language? Devise a mechanism for cross-package-manager interoperaction?... Ridiculous. An application/executable manager should be language agnostic (and not even require compilation). What dub should be first and foremost is a structured build tool (and build specification) for D projects. There is a quite extensive discussion on topic here: http://forum.rejectedsoftware.com/groups/rejectedsoftware.dub/thread/274/ (~ second half), you may want to help me push that idea :P
Re: dub: should we make it the de jure package manager for D?
On Thursday, 12 September 2013 at 20:18:35 UTC, Jonathan M Davis wrote: On Thursday, September 12, 2013 22:02:15 w0rp wrote: On the subject of YAML. I've tried it out a few times. The syntax for JSON can be described in 1-2 pages, on json.org's front page. The syntax for YAML requires a 50-100 page document to describe. That should be enough to tell you not to use it. While, I agree in principle, I would point out that what's on json.org is oversimplified and is not a valid spec. You really need to read the RFC if you want to get it right, and that has about 9 pages of actual content: http://www.ietf.org/rfc/rfc4627.txt?number=4627 So, you're comparing a basic explanation of JSON to a full-on spec for YAML. Now, the YAML 1.2 spec does have about 77 pages of actual content: http://yaml.org/spec/1.2/spec.pdf So, you're not completely off base in your comparison (YAML _is_ way more complicated), but to properly specify JSON requires a lot more pages than your post indicates. - Jonathan M Davis That's a much more fair comparison. Thanks.
Re: dub: should we make it the de jure package manager for D?
On Tuesday, 10 September 2013 at 20:48:58 UTC, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Several remarks (I'm a dub newbie, so it is mostly conditional stuff) : - Can we manage repositories ? For instance can I use my company repo instead of code.dlang.org ? Can I use several repositories with some kind of priorisation ? - How does the tool handle several version of a package ? - What is the process to get some code on code.dlang.org ? We must settle something if this have to go official. - What is the dependance management strategy used ? How does it resolve conflicts ?
Re: dub: should we make it the de jure package manager for D?
Am 13.09.2013 12:00, schrieb deadalnix: On Tuesday, 10 September 2013 at 20:48:58 UTC, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Several remarks (I'm a dub newbie, so it is mostly conditional stuff) : - Can we manage repositories ? For instance can I use my company repo instead of code.dlang.org ? Can I use several repositories with some kind of priorisation ? It supports specifying a list of registries that is searched linearly for packages in its config file (or as a command line argument) - I think I still need to document that, though. The registry currently supports only GitHub and Bitbucket repositories to source packages from, but that can be easily extended by implementing the Repository interface. - How does the tool handle several version of a package ? In the current registry implementation, git tags of the form vX.Y.Z[postfix] are taken as the available versions. When specifying dependencies it's then possible to use relational operators to specify a range of acceptable versions, of which the newest is used. Things like the version pinning mechanism proposed by Jacob can still improve things in this area. - What is the process to get some code on code.dlang.org ? We must settle something if this have to go official. My idea (the status quo) is to keep that as open as possible. Anyone can register an account and register his/her repository. Package names are reserved on a first-come-first-serve basis. This process would then be augmented with community votes, automatic package deprecation and sporadic moderation to keep everything clean in the long term. - What is the dependance management strategy used ? How does it resolve conflicts ? There are basically three areas where things can conflict AFAICS: - Accepted version ranges for packages that are referenced by multiple other packages: Here the best version that satisfies all given ranges is used if possible, otherwise error out - Selected configuration for a package: This is resolved to the first possible set of configurations if possible, otherwise error out (this is a bit more complex due to the platform playing a role in configuration selection) - Conflicts in code (i.e. module/package name clashes) - This is currently not handled, but there have been discussions about that topic: http://forum.rejectedsoftware.com/groups/rejectedsoftware.dub/thread/1/
Re: dub: should we make it the de jure package manager for D?
On Friday, 13 September 2013 at 11:55:42 UTC, Sönke Ludwig wrote: - What is the process to get some code on code.dlang.org ? We must settle something if this have to go official. My idea (the status quo) is to keep that as open as possible. Anyone can register an account and register his/her repository. Package names are reserved on a first-come-first-serve basis. This process would then be augmented with community votes, automatic package deprecation and sporadic moderation to keep everything clean in the long term. I think AUR (Arch Linux Repository) model is very successful one and worth paying attention to when it comes to source packages. Other than voting it also has `out-of-date` flagging by users which helps to noticed abandoned packages fast. There is also small group of trusted users who processes requests for package ownership transfer / package merging via mail list. Any suspicious / malicious package get processed in a similar way. Other than that it is very anarchic environment and still works damn well - AUR package count is now more than 40 000 and Arch Linux is pretty young distro.
Re: dub: should we make it the de jure package manager for D?
On Wed, 11 Sep 2013 15:50:52 -0700 H. S. Teoh hst...@quickfur.ath.cx wrote: On Wed, Sep 11, 2013 at 05:11:38PM -0400, Nick Sabalausky wrote: Seriously, this is just like writing assembly code in 1975. Screw up a single opcode, and the computer just blindly barges onward interpreting random bytes as instructions, wreaking havoc to your entire system while you stare at the screen believing that it's still computing what you think you told it to. Who knew that JS was an underhanded way of getting people to write assembly code by hand again? :-P Exactly. I really do believe that's a very accurate way to put it. I don't like the prevailing notion that such dynamic languages have strong typing. Yes, they *technically* have strong typing, but most benefits are thrown right out the window. So *in effect* it's mostly the same as not having strong typing at all - either way you're still getting tons of seemingly random, unpredictable, barely-tracable, and *potentially dangerous* behavior any time the slightest thing goes wrong. Just because a language has addressed dangerous low-level behaviors doesn't mean it's addressed dangerous high-level behaviors. All they've really accomplished is make the same old problems and catastrophes operate on a higher level than before (but more slowly and with greater power drain). So like you said: Reliability-wise, they're just reinventing assembly. Whitespace is only sometimes significant in YAML. On the JSON constructs, it's not significant. On certain (all?) of the non-JSON YAML-specific things, then it's significant. Yikes. So whitespace is neither always significant nor always insignificant, but *sometimes* significant? Then I have to agree with Jonathan that YAML is truly evil! I don't know about evil. While I don't like indent syntax (it is one of the things I dislike about Python), YAML does at least make it *possible* to avoid both the syntax-fiesta of JSON and the anti-DRYness of XML, and YAML files *can* be constructed to be very easy to read. So I think that alone means YAML deserves at least *some* respect. However, the problem I have with YAML is that it's complex, with various difficult-to-remember syntaxes. I'd feel that I'd have to be very, very careful when writing, editing or generating YAML. And yea, the sometimes significant indenting is certainly part of that complexity. Fun fact: Did you know Python's indenting is only sometimes significant, too?
Re: dub: should we make it the de jure package manager for D?
On 2013-09-12 00:50, H. S. Teoh wrote: But this is JS we're talking about. What happens, boys and gals, when the JS interpreter encounters an uncaught error? That's right, it logs an error message to the error console (which is hidden by default in most browsers), and silently fails without any warning, and then the rest of the site's scripts (hooked to various event handlers) CONTINUE RUNNING AS IF NOTHING HAPPENED. My experience is that if a JavaScript fails, somehow, JavaScript fill stop working all together on the site. But that's perhaps not the case if an exception is thrown. But rather if you're trying to use a variable or similar that doesn't exist. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Thu, Sep 12, 2013 at 08:39:11AM +0200, Jacob Carlborg wrote: On 2013-09-12 00:50, H. S. Teoh wrote: But this is JS we're talking about. What happens, boys and gals, when the JS interpreter encounters an uncaught error? That's right, it logs an error message to the error console (which is hidden by default in most browsers), and silently fails without any warning, and then the rest of the site's scripts (hooked to various event handlers) CONTINUE RUNNING AS IF NOTHING HAPPENED. My experience is that if a JavaScript fails, somehow, JavaScript fill stop working all together on the site. But that's perhaps not the case if an exception is thrown. But rather if you're trying to use a variable or similar that doesn't exist. [...] Well, you may be right, but at the time I was working on a YUI-based framework, and I discovered that either YUI, or the way the framework initializes YUI (I didn't write the base framework code so I don't know for sure), has some kind of default exception-catching code that catches such exceptions and then proceeds to *ignore* it. It does terminate the current execution thread, mind you, but all the other event handlers and hooks are still active, and the rest of the code attached to them will continue running despite the fact that something has catastrophically failed. Which, of course, means that now some object(s) are in an invalid state due to the previous failure, but the code is completely unaware of this situation and continue barging ahead and doing stuff, until it encounters the bad objects, and then random failures happen (which all get swallowed by the default catcher, thus promulgating the problem). But either way, the behaviour is equally bad. If JS stops executing upon encountering an exception, then you just randomly get a blank page (when the bug is triggered) with no helpful indication whatsoever what went wrong. If it continues executing, then you get random failures for no apparent reason. Both are equally hard to debug, and both could've been avoided had JS had *sane* handling of errors in the first place. T -- Gone Chopin. Bach in a minuet.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-12 15:37, H. S. Teoh wrote: Well, you may be right, but at the time I was working on a YUI-based framework, and I discovered that either YUI, or the way the framework initializes YUI (I didn't write the base framework code so I don't know for sure), has some kind of default exception-catching code that catches such exceptions and then proceeds to *ignore* it. It does terminate the current execution thread, mind you, but all the other event handlers and hooks are still active, and the rest of the code attached to them will continue running despite the fact that something has catastrophically failed. Which, of course, means that now some object(s) are in an invalid state due to the previous failure, but the code is completely unaware of this situation and continue barging ahead and doing stuff, until it encounters the bad objects, and then random failures happen (which all get swallowed by the default catcher, thus promulgating the problem). But either way, the behaviour is equally bad. If JS stops executing upon encountering an exception, then you just randomly get a blank page (when the bug is triggered) with no helpful indication whatsoever what went wrong. If it continues executing, then you get random failures for no apparent reason. Both are equally hard to debug, and both could've been avoided had JS had *sane* handling of errors in the first place. Absolutely, I agree. I have seen JavaScript errors on Github from time to time. But it seems like everything continue to work as expected. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
We're considering making dub the official package manager for D. What do you all think? Please let's do it. At first I was like 'why would I need that?'. But soon I ported all my (alive) projects to dub and never came back. Since it can also generate projects for IDEs, it's also a great simplifier of projects. I've noted that it dramatically increased my likeliness to use and discover other's packages.
Re: dub: should we make it the de jure package manager for D?
I am all in favor of making dub *the* D package manager to use. pypi is awesome, and we've got something which can be just as awesome. Let's do it. On the subject of YAML. I've tried it out a few times. The syntax for JSON can be described in 1-2 pages, on json.org's front page. The syntax for YAML requires a 50-100 page document to describe. That should be enough to tell you not to use it.
Re: dub: should we make it the de jure package manager for D?
On Thursday, September 12, 2013 22:02:15 w0rp wrote: On the subject of YAML. I've tried it out a few times. The syntax for JSON can be described in 1-2 pages, on json.org's front page. The syntax for YAML requires a 50-100 page document to describe. That should be enough to tell you not to use it. While, I agree in principle, I would point out that what's on json.org is oversimplified and is not a valid spec. You really need to read the RFC if you want to get it right, and that has about 9 pages of actual content: http://www.ietf.org/rfc/rfc4627.txt?number=4627 So, you're comparing a basic explanation of JSON to a full-on spec for YAML. Now, the YAML 1.2 spec does have about 77 pages of actual content: http://yaml.org/spec/1.2/spec.pdf So, you're not completely off base in your comparison (YAML _is_ way more complicated), but to properly specify JSON requires a lot more pages than your post indicates. - Jonathan M Davis
Re: dub: should we make it the de jure package manager for D?
Am 10.09.2013 23:04, schrieb Nick Sabalausky: On Tue, 10 Sep 2013 23:01:12 +0200 Brad Anderson e...@gnuk.net wrote: I vote yes but only if Sönke feels it is ready. I suspect he has a few things he'll probably want done before this happens (the potential switch from JSON to SDL comes to mind). I assume that would be a backwards-compatible change. Make SDL the preferred, but keep JSON in service. Exactly. Given enough interest, we could also make a more formal review process for a future SDL based format to ensure a maximum chance of a solid, forward compatible format. Of my former list mentioned in the VisualD thread [1], only package signing is really still missing, but that's probably not mission critical for now. The command line build process also needs to be improved one way or another at some point (mostly caching pre-compiled dependencies), but that also isn't really a strong argument anymore. All in all I'd say that the things that are in the package format [2] by now form a pretty solid basis to move forward without worrying too much about future breakage. [1]: http://forum.rejectedsoftware.com/groups/rejectedsoftware.vibed/post/79 [2]: http://code.dlang.org/package-format
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 00:30, schrieb Brad Anderson: On Tuesday, 10 September 2013 at 22:06:28 UTC, luminousone wrote: And license acknowledgement, this is much more important with source libraries then it is with say apt on Ubuntu. Accidentally polluting a bsd project or a closed source project with LGPL/GPL code would be very bad, And often these package management systems obscure what the sources license actually is by simply not showing it or informing the user. There is a license field in the dub package file. It would be neat if dub could warn you if it found incompatible license in your dependencies though. I've added two enhancement requests for this: https://github.com/rejectedsoftware/dub/issues/117 https://github.com/rejectedsoftware/dub-registry/issues/14
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 00:06, schrieb luminousone: Projects that haven't had an update for an excessive amount of time should likely be hidden but still available except in cases where it is known to be unchanged without need for updates(such as most wrappers). Agreed. Maybe some other knowledge, such as how many other (active) packages depend on it, or how often it is still downloaded, can help to get a robust automatic measure.
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 01:17, schrieb Peter Williams: On 11/09/13 06:48, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Is there a 64 bit rpm for dub? Peter AFAIK nobody has started to make RPM's yet (only DEB, AUR and Homebrew so far). Manual installation is trivial though (just ./build.sh and symlink the executable to /usr/local/bin or similar, or put it in PATH).
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 02:18, schrieb Martin Nowak: On 09/10/2013 10:48 PM, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Andrei I think the package format is really good. The registry is essential but needs to support categories, searching and some sort of quality ranking (voting?) for future grow. Full ACK, this is quite critical IMO as the raw package list already provides less than an optimal overview. There are enhancement requests for these in the dub-registry issue tracker [1]. If anyone wants to help out, this is probably one of the most rewarding areas right now (I'll eventually get to it, but things are very busy here). [1]: https://github.com/rejectedsoftware/dub-registry/issues
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 03:49, schrieb Nathan M. Swan: On Tuesday, 10 September 2013 at 20:48:58 UTC, Andrei Alexandrescu wrote: We've been experimenting with http://code.dlang.org for a while and things are going well. In particular Sönke has been very active about maintaining and improving it, which brings further confidence in the future of the project. We're considering making dub the official package manager for D. What do you all think? Andrei I'm all in favor. Careful about the dependencies though, particularly for dub-registry. It's dependent on userman and vibe-d, which is dependent on openssl, libevent, and libev (in Deimos). That's right, at least for DUB itself, all external dependencies have been eliminated, though (except Phobos/libcurl). For the registry site, however, it would be impractical to remove the vibe-d dependency (means a complete rewrite with some other dependency used instead). But should it ever become an issue, the userman dependency could be replaced more easily.
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 06:06, schrieb Jason den Dulk: On Tuesday, 10 September 2013 at 20:48:58 UTC, Andrei Alexandrescu wrote: We're considering making dub the official package manager for D. What do you all think? I think it is a good idea. Having a broad library available for developers to use is a big boost to productivity. However, I agree with luminousone that there need to be some rules about inclusion in the registry. Here are my ideas. 1) Must be legal. 2) Must be release usable. 3) Always has an active caretaker. 5) Have a clear precise descrption of what it does. I think most of this would best be handled by the community using some form of voting/commenting system (with the option for moderation in case of 1) or by automatically fading out packages that fail certain metrics over time (last update to long ago, to few recent downloads, too few other packages depending on it etc.). 4) Must compile and run with a reasonably recent version of the official compiler. Having an integrated CI solution would not only solve 4, but would also allow things such as automatic online documentation for each package. But for so many packages this will of course be difficult in terms of available hardware power and security issues. 6) Have proper licensing. This should be reasonably automated by enforcing that a proper license field is in place and enforcing proper license nesting for known license types (see [1] and [2]). [1]: https://github.com/rejectedsoftware/dub-registry/issues/14 [2]: https://github.com/rejectedsoftware/dub/issues/117
Re: dub: should we make it the de jure package manager for D?
On 2013-09-11 08:28, Sönke Ludwig wrote: Agreed. Maybe some other knowledge, such as how many other (active) packages depend on it, or how often it is still downloaded, can help to get a robust automatic measure. How many total downloads would be nice as well. In RubyGems there are often similar packages and it can be hard to choose which to use. Example: * bootstrap-will_paginate. 257 475 downloads * will_paginate-bootstrap. 84 776 downloads Obviously I'm going to choose the one with most downloads, if there are any critical differences. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-11 08:56, Sönke Ludwig wrote: Having an integrated CI solution would not only solve 4, but would also allow things such as automatic online documentation for each package. But for so many packages this will of course be difficult in terms of available hardware power and security issues. Just use Travis CI, or similar. Unfortunately Travis CI currently only supports Linux and Mac OS X. It doesn't support multiple platforms for a single project. http://travis-ci.org -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-11 06:06, Jason den Dulk wrote: 1) Must be legal. What exactly does this mean in this context? 4) Must compile and run with a reasonably recent version of the official compiler. I think it's better to specify a compiler and version in the package file. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On 2013-09-10 22:48, Andrei Alexandrescu wrote:
We've been experimenting with http://code.dlang.org for a while and
things are going well. In particular Sönke has been very active about
maintaining and improving it, which brings further confidence in the
future of the project.
We're considering making dub the official package manager for D. What do
you all think?
Unfortunately I have to say no to its current state.
The biggest issue I have with dub is that it's really doesn't install
packages, at least not in the traditional sense. I cannot just run dub
install foo and then foo --help. It will only clone the repository,
not install, or install anything. It basically only supports source
packages, which makes it mostly useless for tools/application compiling
to executables.
I would say, compiling and installing executables is a must. It would be
nice if it could compiling libraries as well.
Some other minor issues:
* The registry automatically tracks the git repository. If I register a
new project it will default to master if no tags are available. I
would prefer to tell the registry myself what's available.
* By default a package has the target type autodetect which will try
and build an application and a library, as far as I understand it. Many
projects cannot be built as an application, they're just libraries. Or
the other way around.
* When running dub init foo you get a directory structure like this:
foo
|
|__ public
|
|__ source
| |
| |__ app.d
|
|__ views
|
|__ package.json
This directory structure is very centered around vibe.d. This was fine
when dub was primary the package manger for vibe.d, but I don't think it
should look like this if it becomes the default package manager for D. I
mean, public and views, why would I need those. As I understand it
views can be used for string imports, in that case it would be better
to call it res or resources instead. Most non vibe.d projects would
probably not need this at all so I'm not sure if this should be default.
* Tries to be a build tool and a package manager and at same time
* I'm not sure how it installs indirect dependencies. I'm suspecting it
will always install the latest version of an indirect dependency if
nothing else is specified, which I think is really bad for systems in a
production environment. When the project is built it should locked down
and all the versions of the dependencies, including indirect dependencies.
Say I have a package file looking like this:
{
name: myproject,
dependencies: {
foo: 0.0.1
}
}
And
{
name: foo,
dependencies: {
bar: = 0.0.1
}
}
And
{
name: bar
dependencies: {
}
}
The latest version of bar is 0.0.1.
When building myproject it should lock down bar to the latest
version matching the requirement, that is 0.0.1.
If a new version of bar is out, say 0.0.2, and I'm then installing
myproject on a different computer, I should get the exact same
packages, that is bar should be at version 0.0.1.
If I want a later version of bar I should explicitly tell dub that.
In the end I think it's great if we get a package manager for D. But in
its current state I'm not overly enthusiastic about dub. But depending
on what other thinks and Sönke is willing to take some of this into
consideration I think it could be a good addition to the D tools.
--
/Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 06:28:30 UTC, Sönke Ludwig wrote: Am 11.09.2013 00:06, schrieb luminousone: Projects that haven't had an update for an excessive amount of time should likely be hidden but still available except in cases where it is known to be unchanged without need for updates(such as most wrappers). Agreed. Maybe some other knowledge, such as how many other (active) packages depend on it, or how often it is still downloaded, can help to get a robust automatic measure. Simple idea: try to build the package via current DMD. If compilation false then the package too old (or we have DMD regression). So, it would be nice to have package autotester like for DMD/Phobos repositories.
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 08:51:30 UTC, Jacob Carlborg wrote: On 2013-09-11 06:06, Jason den Dulk wrote: 1) Must be legal. What exactly does this mean in this context? You cannot include anything you do not have the legal right to include. I.E no copyright violations, no child porn software etc. It seems obvious, but it does need to be stated and is a standard part of any terms and conditions related to hosting and distribution. 4) Must compile and run with a reasonably recent version of the official compiler. I think it's better to specify a compiler and version in the package file. Yes, but if the latest version the package is known to work with is more than 3 years old, it would be desirable to have that kept away from the up to date packages.
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 11:57, schrieb Jacob Carlborg:
On 2013-09-10 22:48, Andrei Alexandrescu wrote:
We've been experimenting with http://code.dlang.org for a while and
things are going well. In particular Sönke has been very active about
maintaining and improving it, which brings further confidence in the
future of the project.
We're considering making dub the official package manager for D. What do
you all think?
Unfortunately I have to say no to its current state.
The biggest issue I have with dub is that it's really doesn't install
packages, at least not in the traditional sense. I cannot just run dub
install foo and then foo --help. It will only clone the repository,
not install, or install anything. It basically only supports source
packages, which makes it mostly useless for tools/application compiling
to executables.
I would say, compiling and installing executables is a must. It would be
nice if it could compiling libraries as well.
Right now it is a pure development tool. It would be very nice to have
end user installs somehow supported (either by directly installing
application packages or by generating OS specific packages such as DEB
or RPM). But since this enters a highly operating specific area and goes
into direct competition with the OS package manager, I think it needs a
lot of thought and caution to be generally useful and not possibly do
more harm than good in the end. But yes, it should be a primary goal in
my opinion, too.
Some other minor issues:
* The registry automatically tracks the git repository. If I register a
new project it will default to master if no tags are available. I
would prefer to tell the registry myself what's available.
Why not _make_ a tag? But uploading zipped packages (or better
specifying an external link) could be added as an alternative without
much effort.
* By default a package has the target type autodetect which will try
and build an application and a library, as far as I understand it. Many
projects cannot be built as an application, they're just libraries. Or
the other way around.
This is something that may still need some adjustments, but after all
it's just a default when there is a source/app.d or
source/packname.d file - an explicit targetType: xxx will fix it
when it's off.
* When running dub init foo you get a directory structure like this:
foo
|
|__ public
|
|__ source
| |
| |__ app.d
|
|__ views
|
|__ package.json
This directory structure is very centered around vibe.d. This was fine
when dub was primary the package manger for vibe.d, but I don't think it
should look like this if it becomes the default package manager for D. I
mean, public and views, why would I need those. As I understand it
views can be used for string imports, in that case it would be better
to call it res or resources instead. Most non vibe.d projects would
probably not need this at all so I'm not sure if this should be default.
views is maybe not general enough, but for what it's worth I'm also
using it on other kinds of projects (e.g. to specify GUI templates).
res might be a bit *too* general, but this could for sure be improved
somehow. public is indeed specific for web stuff, so it may be the
best to drop it by default.
I've also thought about offering a set of template projects usable for
dub init, so that there could e.g. be a true vibe.d skeleton, but by
default it would be a minimal project with only a source folder.
* Tries to be a build tool and a package manager and at same time
Note that there is dub describe to extract the information relevant
for actual building plus there is dub generate xxx, so nothing is in
the way of using a separate tool for building. But having integrated
building IMO is highly convenient and keeping the build description in a
standardized format is one of the key selling points.
* I'm not sure how it installs indirect dependencies. I'm suspecting it
will always install the latest version of an indirect dependency if
nothing else is specified, which I think is really bad for systems in a
production environment. When the project is built it should locked down
and all the versions of the dependencies, including indirect dependencies.
Say I have a package file looking like this:
{
name: myproject,
dependencies: {
foo: 0.0.1
}
}
And
{
name: foo,
dependencies: {
bar: = 0.0.1
}
}
And
{
name: bar
dependencies: {
}
}
The latest version of bar is 0.0.1.
When building myproject it should lock down bar to the latest
version matching the requirement, that is 0.0.1.
If a new version of bar is out, say 0.0.2, and I'm then installing
myproject on a different computer, I should get the exact same
packages, that is bar should be at version 0.0.1.
If I want a later version of bar I should explicitly tell dub that.
You can put an additional bar: ==0.0.1 dependency in the main
package to lock
Re: dub: should we make it the de jure package manager for D?
On 11-9-2013 12:28, ilya-stromberg wrote: On Wednesday, 11 September 2013 at 06:28:30 UTC, Sönke Ludwig wrote: Am 11.09.2013 00:06, schrieb luminousone: Projects that haven't had an update for an excessive amount of time should likely be hidden but still available except in cases where it is known to be unchanged without need for updates(such as most wrappers). Agreed. Maybe some other knowledge, such as how many other (active) packages depend on it, or how often it is still downloaded, can help to get a robust automatic measure. Simple idea: try to build the package via current DMD. If compilation false then the package too old (or we have DMD regression). So, it would be nice to have package autotester like for DMD/Phobos repositories. This way it actually knows the last compatible dmd version.
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 11:31:11 UTC, Faux Amis wrote: On 11-9-2013 12:28, ilya-stromberg wrote: On Wednesday, 11 September 2013 at 06:28:30 UTC, Sönke Ludwig wrote: Am 11.09.2013 00:06, schrieb luminousone: Projects that haven't had an update for an excessive amount of time should likely be hidden but still available except in cases where it is known to be unchanged without need for updates(such as most wrappers). Agreed. Maybe some other knowledge, such as how many other (active) packages depend on it, or how often it is still downloaded, can help to get a robust automatic measure. Simple idea: try to build the package via current DMD. If compilation false then the package too old (or we have DMD regression). So, it would be nice to have package autotester like for DMD/Phobos repositories. This way it actually knows the last compatible dmd version. Yes. And list of problem packages with compilation errors or broken dependencies.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-11 12:28, ilya-stromberg wrote: Simple idea: try to build the package via current DMD. If compilation false then the package too old (or we have DMD regression). So, it would be nice to have package autotester like for DMD/Phobos repositories. Why should you be forced to constantly use the latest DMD? That can be a lot of work for the package developer for no gain. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 12:26:05 UTC, Jacob Carlborg wrote: On 2013-09-11 12:28, ilya-stromberg wrote: Simple idea: try to build the package via current DMD. If compilation false then the package too old (or we have DMD regression). So, it would be nice to have package autotester like for DMD/Phobos repositories. Why should you be forced to constantly use the latest DMD? That can be a lot of work for the package developer for no gain. Because it is current D reality. Package that do not get updated to latest front-end version are used only if there is absolutely no other choice. Amount of inconvenience it causes to the user of the package is tremendous.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-11 13:03, Jason den Dulk wrote: Yes, but if the latest version the package is known to work with is more than 3 years old, it would be desirable to have that kept away from the up to date packages. Three years is a bit different. I'm thinking more that it need to support multiple versions of the compilers because DMD breaks code in every single release. Why should I update a package that doesn't take advantage of any new features or bug fixes? It's just maintenance work. -- /Jacob Carlborg
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 09:57:48 UTC, Jacob Carlborg wrote: The biggest issue I have with dub is that it's really doesn't install packages, at least not in the traditional sense. I cannot just run dub install foo and then foo --help. It will only clone the repository, not install, or install anything. It basically only supports source packages, which makes it mostly useless for tools/application compiling to executables. I would say, compiling and installing executables is a must. It would be nice if it could compiling libraries as well. I am strongly against it. It is not a job of language package manager. Implementing it properly will require to integrate the knowledge of every existing packaging system among every slightly popular OS / distro. Implement it as a hack with own package ecosystem and people will hate you. It exists to simplify development. Users that run D-based programs should never ever be even aware of such thing as `dub`.
Re: dub: should we make it the de jure package manager for D?
On 9/11/13 5:01, Brad Anderson wrote: I vote yes but only if Sönke feels it is ready. I suspect he has a few things he'll probably want done before this happens (the potential switch from JSON to SDL comes to mind). SD-what?! Why would alienate people even more than we already do? L.
Re: dub: should we make it the de jure package manager for D?
On Wed, 11 Sep 2013 21:10:46 +0800
Lionello Lunesu lione...@lunesu.remove.com wrote:
On 9/11/13 5:01, Brad Anderson wrote:
I vote yes but only if Sönke feels it is ready. I suspect he has a
few things he'll probably want done before this happens (the
potential switch from JSON to SDL comes to mind).
SD-what?! Why would alienate people even more than we already do?
L.
{
name: myproject,
description: A little web service of mine.,
authors: [Peter Parker, Joe Contrib],
dependencies: {
vibe-d: =0.7.11,
mylib:component1: ~master,
mylib:component2: ~master
},
subPackages: [
{
name: component1,
targetType: library,
sourcePaths: [source/component1]
},
{
name: component2,
targetType: library,
sourcePaths: [source/component2]
}
]
}
vs:
name myproject,
description A little web service of mine.
authors Peter Parker Joe Contrib
dependencies {
vibe-d =0.7.11
mylib:component1 ~master
mylib:component2 ~master
}
subPackage {
name component1
targetType library
sourcePaths source/component1
}
subPackage {
name component2
targetType library
sourcePaths source/component2
}
That's why.
Besides, the JSON form wouldn't be going away anyway, it'd still be
kept.
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 13:39:02 UTC, Nick Sabalausky
wrote:
On Wed, 11 Sep 2013 21:10:46 +0800
Lionello Lunesu lione...@lunesu.remove.com wrote:
On 9/11/13 5:01, Brad Anderson wrote:
I vote yes but only if Sönke feels it is ready. I suspect he
has a
few things he'll probably want done before this happens (the
potential switch from JSON to SDL comes to mind).
SD-what?! Why would alienate people even more than we already
do?
L.
{
name: myproject,
description: A little web service of mine.,
authors: [Peter Parker, Joe Contrib],
dependencies: {
vibe-d: =0.7.11,
mylib:component1: ~master,
mylib:component2: ~master
},
subPackages: [
{
name: component1,
targetType: library,
sourcePaths: [source/component1]
},
{
name: component2,
targetType: library,
sourcePaths: [source/component2]
}
]
}
vs:
name myproject,
description A little web service of mine.
authors Peter Parker Joe Contrib
dependencies {
vibe-d =0.7.11
mylib:component1 ~master
mylib:component2 ~master
}
subPackage {
name component1
targetType library
sourcePaths source/component1
}
subPackage {
name component2
targetType library
sourcePaths source/component2
}
That's why.
Besides, the JSON form wouldn't be going away anyway, it'd
still be
kept.
Why not YAML? It's cleaner than JSON and is very widely known.
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 15:01:37 UTC, Jacob Carlborg wrote: I'm thinking this type of package manager should be a development tool as well. But there are a lot of development tools that are executables and not just libraries. Think of your documentation generator. Without having looked at it I would assume it's an executable. I have myself a tool, DStep, which translate C headers to D modules. This is an executable as well. Those should be provided as sources and built by dub too. Distributing binary packages requires both package signing and reasonable web of trust - something that is not easy to just implement from scratch. Otherwise any single malicious package may ruin reputation of the whole system.
Re: dub: should we make it the de jure package manager for D?
On 2013-09-11 13:30, Sönke Ludwig wrote: Right now it is a pure development tool. It would be very nice to have end user installs somehow supported (either by directly installing application packages or by generating OS specific packages such as DEB or RPM). But since this enters a highly operating specific area and goes into direct competition with the OS package manager, I think it needs a lot of thought and caution to be generally useful and not possibly do more harm than good in the end. But yes, it should be a primary goal in my opinion, too. I'm thinking this type of package manager should be a development tool as well. But there are a lot of development tools that are executables and not just libraries. Think of your documentation generator. Without having looked at it I would assume it's an executable. I have myself a tool, DStep, which translate C headers to D modules. This is an executable as well. Why not _make_ a tag? But uploading zipped packages (or better specifying an external link) could be added as an alternative without much effort. I'm not referring to uploading zip packages. Say I have ten tags but I only want dub to know about five of them. I want to manually say make the tag v0.0.1 available. This is something that may still need some adjustments, but after all it's just a default when there is a source/app.d or source/packname.d file - an explicit targetType: xxx will fix it when it's off. Yes, but I still don't like the default. To avoid misunderstandings, will the dub, by default _both_ build an executable _and_ a library. Or will it build an executable _or_ a library? views is maybe not general enough, but for what it's worth I'm also using it on other kinds of projects (e.g. to specify GUI templates). res might be a bit *too* general, but this could for sure be improved somehow. public is indeed specific for web stuff, so it may be the best to drop it by default. It's possible to import images, video, audio and many other type of resources as well using string imports. I've also thought about offering a set of template projects usable for dub init, so that there could e.g. be a true vibe.d skeleton, but by default it would be a minimal project with only a source folder. Yes, I was thinking the same. Note that there is dub describe to extract the information relevant for actual building plus there is dub generate xxx, so nothing is in the way of using a separate tool for building. But having integrated building IMO is highly convenient and keeping the build description in a standardized format is one of the key selling points. Hmm ok. I would go with two separate tools that are well integrated with each other. BTW, neither dub generate rdmd or dub generate build seems to be working. You can put an additional bar: ==0.0.1 dependency in the main package to lock it to a certain version. I don't want to care about indirect dependencies. That's the job of the package manager. If I need to keep track of indirect dependencies then the package manager isn't helping much. Or, of course, just use == in the bar package in the first place. I might only control the myproject package. This could also be implemented as a (semi-)automatic function along the lines of dub lock-versions and dub lock-versions --upgrade. I think this should be the default. I don't want my package to break at random just because there's a newer version available of an indirect dependency. I think that a key feature of a package manager should be that any time in time you're installing a package of a given version should result in the _exact_ same packages, including indirect packages. On the other hand it's difficult to make a general statement that this is always the best way as this may for example prevent important security fixes to get incorporated unless the main package maintainer releases an explicit update... Any update, may it only be security fixes, may break a build. No, semantic versions doesn't help here. Locking down by default is the safest. If the package maintainer doesn't release an update how will you get security updates? It's better if there's an explicit way to update an indirect dependency. I would rather add bar: ==0.0.2 on one or two packages to force an update of an indirect dependency rather then set bar: ==0.0.1 on _all_ indirect dependency. This can be quite a large tree if there are many dependencies. At work we have a Ruby on Rails project. It currently uses 160 packages, including Rails and all its sub components. It was pure hell before bundler was released which locks down all package dependencies as I've described here. Counting all indirect dependencies as well I get over 413 packages. Would you like to specify the exact version of all these? Saying that bundler was a salvation for the Ruby community is an understatement. When we get a new employee or a new computer we want, of course, to
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 14:11:11 UTC, John Colvin wrote: Why not YAML? It's cleaner than JSON and is very widely known. YAML is nice but can be surprisingly tricky to write by hand sometimes (especially for people not used to significant whitespace). Here's the discussion about JSON vs. YAML vs. SDL on the dub forum: http://forum.rejectedsoftware.com/groups/rejectedsoftware.dub/thread/2/
Re: dub: should we make it the de jure package manager for D?
Am 11.09.2013 17:01, schrieb Jacob Carlborg: Why not _make_ a tag? But uploading zipped packages (or better specifying an external link) could be added as an alternative without much effort. I'm not referring to uploading zip packages. Say I have ten tags but I only want dub to know about five of them. I want to manually say make the tag v0.0.1 available. It will only look at version tags of the form vA.B.C(postfix) any reason to hide one of those? It could be added as a feature to the registry, but is there a compelling use case to warrant the costs? This is something that may still need some adjustments, but after all it's just a default when there is a source/app.d or source/packname.d file - an explicit targetType: xxx will fix it when it's off. Yes, but I still don't like the default. To avoid misunderstandings, will the dub, by default _both_ build an executable _and_ a library. Or will it build an executable _or_ a library? or, you can choose which using dub --config=library or dub --config=application (the default). Note that there is dub describe to extract the information relevant for actual building plus there is dub generate xxx, so nothing is in the way of using a separate tool for building. But having integrated building IMO is highly convenient and keeping the build description in a standardized format is one of the key selling points. Hmm ok. I would go with two separate tools that are well integrated with each other. BTW, neither dub generate rdmd or dub generate build seems to be working. Works for me, but master was broken for some hours. Maybe you caught a bad version? This could also be implemented as a (semi-)automatic function along the lines of dub lock-versions and dub lock-versions --upgrade. I think this should be the default. I don't want my package to break at random just because there's a newer version available of an indirect dependency. I think that a key feature of a package manager should be that any time in time you're installing a package of a given version should result in the _exact_ same packages, including indirect packages. I just don't know if that can be generalized to everyone or at least the majority of users. Maybe this is a good topic for starting a wider discussion/poll. On the other hand it's difficult to make a general statement that this is always the best way as this may for example prevent important security fixes to get incorporated unless the main package maintainer releases an explicit update... Any update, may it only be security fixes, may break a build. No, semantic versions doesn't help here. Locking down by default is the safest. If used correctly, by definition, it _does_ help. Reality, especially in the fast moving D environment, may be different, though. However it seems to work quite well in the C world. If the package maintainer doesn't release an update how will you get security updates? By upgrading and getting updates of indirect dependencies, even if the main package wasn't updated. It's better if there's an explicit way to update an indirect dependency. I would rather add bar: ==0.0.2 on one or two packages to force an update of an indirect dependency rather then set bar: ==0.0.1 on _all_ indirect dependency. This can be quite a large tree if there are many dependencies. At work we have a Ruby on Rails project. It currently uses 160 packages, including Rails and all its sub components. It was pure hell before bundler was released which locks down all package dependencies as I've described here. Counting all indirect dependencies as well I get over 413 packages. Would you like to specify the exact version of all these? Saying that bundler was a salvation for the Ruby community is an understatement. When we get a new employee or a new computer we want, of course, to have the _exact_ same packages we use on all the other development machines and the ones in production. Otherwise we cannot ensure that everything is working the same way on our development machines as on the production machines. Hey, just adding a new production machine we could end up with different packages before. Pure nightmare. Note that I definitely don't oppose to the idea of integrating such a mechanism -- I do see the value. The question is if it should be the default or not (Bundler is opt-in after all), taking into account for example how well it interacts with branches and different use cases. I'm quite open there, but I first have to play that through with all of my stuff before I can voice an opinion.
Re: dub: should we make it the de jure package manager for D?
On Wednesday, 11 September 2013 at 06:12:41 UTC, Sönke Ludwig wrote: Am 10.09.2013 23:04, schrieb Nick Sabalausky: On Tue, 10 Sep 2013 23:01:12 +0200 Brad Anderson e...@gnuk.net wrote: I vote yes but only if Sönke feels it is ready. I suspect he has a few things he'll probably want done before this happens (the potential switch from JSON to SDL comes to mind). I assume that would be a backwards-compatible change. Make SDL the preferred, but keep JSON in service. Exactly. Given enough interest, we could also make a more formal review process for a future SDL based format to ensure a maximum chance of a solid, forward compatible format. Of my former list mentioned in the VisualD thread [1], only package signing is really still missing, but that's probably not mission critical for now. The command line build process also needs to be improved one way or another at some point (mostly caching pre-compiled dependencies), but that also isn't really a strong argument anymore. All in all I'd say that the things that are in the package format [2] by now form a pretty solid basis to move forward without worrying too much about future breakage. [1]: http://forum.rejectedsoftware.com/groups/rejectedsoftware.vibed/post/79 [2]: http://code.dlang.org/package-format Perhaps there should be a version number in the package format so it can be changed when needed without a lot of headache and breakage.
