Re: Make dub part of the standard dmd distribution
On 2015-06-10 14:11, Sönke Ludwig wrote: Getting some of those use cases on a Wiki page or something would be great. There have been some ideas to approach the procedural aspect from a different angle - using procedural DUB plugins that can be invoked from within the declarative build description. This would have the advantage that a lot of information can be gathered about a package without executing procedural code (with the possible accompanying security risks and performance issues or the efforts needed to mitigate those). Orbit, the package manager I was working on, uses Ruby as the description language. It uses the more traditional way of uploading packages to the registry, basically a ZIP with with a special directory layout with every file necessary to build the package. With this model it'a straight forward to compile the Ruby code to JSON or YAML when building the package locally (same thing can be done with D). On the server side, the registry only needs to read the JSON file to get the metadata of the package. When installing the package it would use the Ruby file again (or JSON description, never got that far), to be able to run pre and post actions. This part shouldn't be any less secure than the current pre/post commands. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
Am 04.06.2015 um 17:47 schrieb Atila Neves: On a related note, it's also a pity that Reggae mixes incremental build improvements (from which DUB itself could greatly profit, too - just as a Ninja generator for DUB would be a nice feature) with a separate, layered build description. I mean there is of course no reason to not have alternative approaches for build descriptions available in general, but when mixed with a public package repository, it just leads to fragmentation. I can add a ninja generator to dub if you want, with the default being per-package compilation (since right now for dub it's one file or everything at once). That would be great! If the per-package mode has such practical benefits, we could also simply make it the general default, for all generators that can support it. However, I just briefly looked at your benchmark results, the performance boost seems to be solely because of parallel compiler invocations. We already had issues with per-module parallel compilation and out-of-memory errors, so this optimization seems to be only suitable for certain projects or build machines. So at least with the current compiler memory requirements I'm not sure if this would be a good default build mode. But I wrote reggae because: 1. Declarative is preferable but imperative is needed. My (our?) favourite imperative language is D, so I want to specify builds in it. This is of course perfectly fine and reasonable (although I personally find the D syntax to be a bit too verbose for this task, but thats really just personal taste - oh, and JSON is awful, too ;). The problems just start to creep in as soon as public DUB packages start to depend on Reggae - then we'll possibly get another split in the ecosystem. I'd say we should generally try to focus on a single standard solution, however that looks and however the tool is split into executables or packages. 2. I want a build tool that builds on dub but doesn't require it Fair enough. 3. I wanted it to make it easy to link with C and C++ This really also needs to be solved within DUB (i.e. without resorting to preBuildCommands). That is going to be the next priority after getting the 1.0.0 features ready. As I've mentioned before, I know the kind of things I'd want to do with the build system if I had a large and complicated enough project, and I know I wouldn't be able to do it easily using dub alone. As I've also mentioned before, building with dub is just fine for most people. Getting some of those use cases on a Wiki page or something would be great. There have been some ideas to approach the procedural aspect from a different angle - using procedural DUB plugins that can be invoked from within the declarative build description. This would have the advantage that a lot of information can be gathered about a package without executing procedural code (with the possible accompanying security risks and performance issues or the efforts needed to mitigate those). I don't know if fragmentation would be an issue. The packages are still dub packages and I for one will use dub.json/sdl to list my dependencies even if reggae is actually generating the build. Yeah that would definitely not be an issue. I just fear (maybe unnecessarily) that people might start to put packages in the registry that can *only* be built using Reggae (or some other build tool). At least for libraries that would really be bad.
Re: Make dub part of the standard dmd distribution
Am 05.06.2015 um 11:56 schrieb Marc =?UTF-8?B?U2Now7x0eiI=?=
schue...@gmx.net:
On Thursday, 4 June 2015 at 13:47:41 UTC, Sönke Ludwig wrote:
A branch *is* an exact version as far as DUB goes (just one that
changes over time).
Well, that's the problem then.
But to make use of any commit relationships within a branch, it would
have to gain specific knowledge of GIT or other version management
systems.
Adding support for commit hashes in dub.selections.json is still TBD,
but would be another alternative.
I'd say it's the only consistent way.
BTW, my primary concern with all of these things is retaining a
healthy public package eco system. I personally suspect that this is
much more important for the language as a whole rather than pleasing
every single taste. This doesn't mean that the latter isn't a
worthwhile goal, but to me the former has always been more important.
What is missing is a proposal how to solve both issues at the same time.
I can understand that. But from what I've seen with rubygems, it is not
a problem in practice. You're totally right that released packages
shouldn't have such dependencies. A warning message in dub or a check on
code.dlang.org is totally okay IMO.
Well, enforcing it on code.dlang.org would indeed be an option. I still
think that using the version backed approach below, or VCS facilities is
superior in general, though. There is not much objective reason to
duplicate that functionality in the package manager AFAICS.
For example for conflict resolution. This is the foremost reason why
the *have* to be banned from dub.json. You can't really know if a
branch matches/is compatible with any particular version (range) or
another branch, so you have to either make an arbitrary guess (always
prefer the branch; always prefer the version; always prefer master,
but a tagged version otherwise; etc.), or always issue a version
conflict error.
Definitely always prefer the branch. The goal is to let the developer
override the decisions made by upstream library authors.
That's what dub.selections.json is for! It applies only to the top level
project, which can make such decisions. On the other hand, if you always
prefer the branch during *dependency resolution*, some deeply buried
dependency can simply break your build (or any upstream build) because
it chose to use an incompatible branch based dependency.
The practical effect that was already very visible in the existing
ecosystem was that packages tended to separate into two clusters. One
cluster that used branch dependencies and another that used version
dependencies. Both incompatible with each other. It also meant that a
lot of people didn't bother to make any version tags at all, which is
understandable, but also a very bad influence to the stability of the
ecosystem. Semantic versioning, if used correctly, is a very important
factor in that regard.
Yes, but that's a social problem. We can nudge people in the right
direction by printing a warning message, or even by rejecting it
completely on code.dlang.org.
The compromise solution to allow branch based dependencies only in the
version selection file doesn't really have drawbacks, but dodges all
of those issues by moving the decision to the root of the dependency
tree.
I does have obvious drawbacks: If dub.selections.json contains non-exact
versions, different developers can get slightly different dependencies.
dub.selections.json must fully determine the versions that are going to
be used, otherwise it has failed its purpose.
You can also check out certain branches locally and use dub
add-local or dub add-path to let DUB use it instead of the publicly
registered one.
This is not an option in a larger team. Everyone would have to do that
manually, or you'd need to write a custom script to do what is actually
the package manager's task.
Using dub.selections.json should be the right tool then.
As explained above, it isn't.
If you want commit granularity then that's a different thing. The
intended way to support this is to still use version tags - DUB will
then associate a branch with the latest matching version tag, so that
normal version based dependencies can be used.
If that case applies, dub.selections.json will contain something like
{somedep: 1.0.0+commit.14.f23a56gb}. Currently this information
isn't used to drive GIT to fetch the right tag, but a ticket for that is
open and this could be a viable middleground solution.
You could of course also use things like git submodules + path based
dependencies in this case, there is no need to force everything into
one schema (especially when it doesn't fit well, such as versions and
branches).
Yes, and I'm sure I can come up with even more workarounds ;-)
It's not really a workaround. The package manager has a version based
dependency handling logic that has the proper semantics for this kind of
task. Trying to squeeze branches and commits into this isn't
Re: Make dub part of the standard dmd distribution
That would be great! If the per-package mode has such practical benefits, we could also simply make it the general default, for all generators that can support it. However, I just briefly looked at your benchmark results, the performance boost seems to be solely because of parallel compiler invocations. We already had issues with per-module parallel compilation and out-of-memory errors, so this optimization seems to be only suitable for certain projects or build machines. So at least with the current compiler memory requirements I'm not sure if this would be a good default build mode. From mine and Andrei's results it seems to be a much better default. This is of course perfectly fine and reasonable (although I personally find the D syntax to be a bit too verbose for this task, but thats really just personal taste - oh, and JSON is awful, too ;). The problems just start to creep in as soon as public DUB packages start to depend on Reggae - then we'll possibly get another split in the ecosystem. I'd say we should generally try to focus on a single standard solution, however that looks and however the tool is split into executables or packages. I'm working on the syntax ;) As I've mentioned before, I know the kind of things I'd want to do with the build system if I had a large and complicated enough project, and I know I wouldn't be able to do it easily using dub alone. As I've also mentioned before, building with dub is just fine for most people. Getting some of those use cases on a Wiki page or something would be great. I'd have to go back to my old work project and read all the CMake code... I don't know if fragmentation would be an issue. The packages are still dub packages and I for one will use dub.json/sdl to list my dependencies even if reggae is actually generating the build. Yeah that would definitely not be an issue. I just fear (maybe unnecessarily) that people might start to put packages in the registry that can *only* be built using Reggae (or some other build tool). At least for libraries that would really be bad. I think that if a project is a dub package, then it needs to be able to be built with dub. If anything on code.dlang.org doesn't (correctly) build with `dub build` then that would be incredibly wrong. Also, I'm not as sure as you that reggae will catch on that much :) Atila
Re: Make dub part of the standard dmd distribution
On 06/10/2015 08:11 AM, Sönke Ludwig wrote: Am 04.06.2015 um 17:47 schrieb Atila Neves: I don't know if fragmentation would be an issue. The packages are still dub packages and I for one will use dub.json/sdl to list my dependencies even if reggae is actually generating the build. Yeah that would definitely not be an issue. I just fear (maybe unnecessarily) that people might start to put packages in the registry that can *only* be built using Reggae (or some other build tool). At least for libraries that would really be bad. I've found the opposite to be true. We can *already* get packages in code.dlang.org that work with a custom buildscript, but then fail when doing dub build. I know this because I've *already* done it by accident multiple times. And then I get bug reports of X or Y build problem under dub. But if we can easily tell dub via dub.json, Hey, don't use your internal build system to build this, I've stuck the appropriate command in pre/postBuildCommands, then dub build will automatically just simply work. Major bonus compatibility points the more information we can pass from dub to the custom command (ex: Which 'built type' did the user request with '--build='? or...well, everything provided by dub describe --data). Remember, libraries do NOT need to use the same build system to be used together. As long as there's a common interface to... 1. Trigger a build, and 2. Get the lists of import paths and output binaries (and maybe any other required flags), ...then everything's golden. This is the job of a package manager. Each library author tells the package manager: 1. When someone asks you to build me, here's how you do it and 2. When someone asks you for my import paths and output binaries (etc.), this is what you tell them For DUB, #1 above can be done via pre/postBuildCommands as long as we can tell it Don't go trying to build this yourself. (I'm wondering if targetType: none would help here.) And for #2, DUB can do that via my --data= pull request. Ok, only one possible issue left: Source libraries (as opposed to binary libraries) *could* be an issue just because of the differences in collecting the list of files to be compiled. But that's solvable two ways: 1. If your lib *requires* something like RDMD-style dependency collecting, then the lib must be a binary lib. Or much, much better yet: 2. Option in dub.json to say when you collect this package's list of source files, do it RDMD-style. Sure, that may not be appropriate for ALL packages, but that's exactly why it'd be a per-package OPTION. And there it is, the *existing* issues of splitting the dub ecosystem are thus eliminated, without creating new splits.
Re: Make dub part of the standard dmd distribution
On 06/10/2015 09:06 AM, Atila Neves wrote: I think that if a project is a dub package, then it needs to be able to be built with dub. If anything on code.dlang.org doesn't (correctly) build with `dub build` then that would be incredibly wrong. Right. And to summarize what I said elsewhere about that: Building WITH dub need NOT imply building using dub's internal buildsystem. It only needs to mean The CORRECT buildsystem can be trigged via 'dub build', whatever buildsystem that may be. That's why the current situation is backwards and CAUSES splits rather than preventing them: We can always trigger dub's internal buildsystem via dub build, but trying to get dub build to trigger the CORRECT one INSTEAD of built-in (whenever a different one is needed) is a world of hurt.
Re: Make dub part of the standard dmd distribution
On Tuesday, 9 June 2015 at 14:09:03 UTC, Dicebot wrote: I don't see how it matters as long as packaging is done properly. Developer should not ever be allowed to write to /usr anyway. +1 I don't expect to be able to rely on the location of anything under /usr except maybe /usr/bin/env and I don't see why D should be any different.
Re: Make dub part of the standard dmd distribution
I don't see how it matters as long as packaging is done properly. Developer should not ever be allowed to write to /usr anyway.
Re: Make dub part of the standard dmd distribution
Am 08.06.2015 um 08:47 schrieb Jacob Carlborg: On 2015-06-08 08:04, Sönke Ludwig wrote: Isn't just the list of *modules* needed and not the list of *imports*? The registry can easily query the list of files of a package using the GitHub API. The package description is obviously already known, too, so this would definitely be possible to do without compiling anything. There's still the issue with a module not having a matching filename. I think that will only work with a target type sourceLibrary anyway (since such modules always have to be on the compiler command line), so it will always be at least somewhat problematic. But there is already some code [1] to extract the module name for dub test, so that would still be easy to do without actually invoking the compiler. [1]: https://github.com/D-Programming-Language/dub/blob/b3ea9e9f026bd096c60b4d1de55e2ea62bd313cd/source/dub/dub.d#L777
Re: Make dub part of the standard dmd distribution
On 9 June 2015 at 15:52, Marco Leise via Digitalmars-d digitalmars-d@puremagic.com wrote: Am Mon, 1 Jun 2015 13:48:21 +1000 schrieb Manu via Digitalmars-d digitalmars-d@puremagic.com: Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff […]. I would like to mention that 40 out of 62 ppl by the end of 2013 did agreed on /usr/include/dlang : http://forum.dlang.org/thread/20131112205019.12585bbd@marco-leise That's a few more than nobody. Today I would include a other choice at least, but it wouldn't have changed the picture much. There was no 'd2' option, which is the only one I've ever encountered. Is there a convention in use? Does anyone else bother with this? I just want someone to decide and publish it somewhere... I don't care what's chosen, but we really need to just agree on something.
Re: Make dub part of the standard dmd distribution
Am 07.06.2015 um 19:57 schrieb Jacob Carlborg: On 2015-06-07 17:27, Nick Sabalausky wrote: Even if that stuff isn't currently known to code.dlang.org, it'd all be easy for it to obtain: $ mkdir some_temp cd some_temp $ git clone package_URL dir cd dir $ dmd (flags dub already knows) -v | grep import Would only need to do that when it detects a new version tag (which it already detects). So the registry basically needs to compile all packages? Isn't it better to compile and upload packages then? Isn't just the list of *modules* needed and not the list of *imports*? The registry can easily query the list of files of a package using the GitHub API. The package description is obviously already known, too, so this would definitely be possible to do without compiling anything.
Re: Make dub part of the standard dmd distribution
On 2015-06-08 08:04, Sönke Ludwig wrote: Isn't just the list of *modules* needed and not the list of *imports*? The registry can easily query the list of files of a package using the GitHub API. The package description is obviously already known, too, so this would definitely be possible to do without compiling anything. There's still the issue with a module not having a matching filename. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2015-06-07 17:27, Nick Sabalausky wrote: Even if that stuff isn't currently known to code.dlang.org, it'd all be easy for it to obtain: $ mkdir some_temp cd some_temp $ git clone package_URL dir cd dir $ dmd (flags dub already knows) -v | grep import Would only need to do that when it detects a new version tag (which it already detects). So the registry basically needs to compile all packages? Isn't it better to compile and upload packages then? -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 06/06/2015 09:16 AM, Jacob Carlborg wrote: On 2015-06-05 15:17, Steven Schveighoffer wrote: The source of the package does. code.dlang.org can create this automatically. I see two problems with this: 1. Does the registry (code.dlang.org) really know which files a Dub package contains? I would guess the registry mostly contains information where to find the source for a package, i.e. GitHub Rubygems, for example, on the other hand would be able to do this because a Ruby gem (package) is ZIP archive with all the sources for the package included. Also, when building a gem the package description is compiled, that is, all shell globs are expanded to locate all files. 2. It's possible to have a file with a completely different module name in D Even if that stuff isn't currently known to code.dlang.org, it'd all be easy for it to obtain: $ mkdir some_temp cd some_temp $ git clone package_URL dir cd dir $ dmd (flags dub already knows) -v | grep import Would only need to do that when it detects a new version tag (which it already detects).
Re: Make dub part of the standard dmd distribution
On 2015-06-05 15:17, Steven Schveighoffer wrote: The source of the package does. code.dlang.org can create this automatically. I see two problems with this: 1. Does the registry (code.dlang.org) really know which files a Dub package contains? I would guess the registry mostly contains information where to find the source for a package, i.e. GitHub Rubygems, for example, on the other hand would be able to do this because a Ruby gem (package) is ZIP archive with all the sources for the package included. Also, when building a gem the package description is compiled, that is, all shell globs are expanded to locate all files. 2. It's possible to have a file with a completely different module name in D -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2015-06-04 19:44, Steven Schveighoffer wrote: The use case I'm thinking of is basically: import somepackage.somemodule; ... compile - can't find somepackage.somemodule. did you mean somepackage.somemodule from libgeneric on code.dlang.org? (url) to add: dub depend add libgeneric dub depend add libgeneric = found it, added it, added dependency I don't have to look up anything on code.dlang.org, I don't have to do anything, dub figures it all out from the description file online. Sounds like a nice feature but how would that work in practice? I mean, there's nothing that connects a particular D module or package to a particular Dub package. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On Thursday, 4 June 2015 at 13:47:41 UTC, Sönke Ludwig wrote: A branch *is* an exact version as far as DUB goes (just one that changes over time). Well, that's the problem then. But to make use of any commit relationships within a branch, it would have to gain specific knowledge of GIT or other version management systems. Adding support for commit hashes in dub.selections.json is still TBD, but would be another alternative. I'd say it's the only consistent way. BTW, my primary concern with all of these things is retaining a healthy public package eco system. I personally suspect that this is much more important for the language as a whole rather than pleasing every single taste. This doesn't mean that the latter isn't a worthwhile goal, but to me the former has always been more important. What is missing is a proposal how to solve both issues at the same time. I can understand that. But from what I've seen with rubygems, it is not a problem in practice. You're totally right that released packages shouldn't have such dependencies. A warning message in dub or a check on code.dlang.org is totally okay IMO. For example for conflict resolution. This is the foremost reason why the *have* to be banned from dub.json. You can't really know if a branch matches/is compatible with any particular version (range) or another branch, so you have to either make an arbitrary guess (always prefer the branch; always prefer the version; always prefer master, but a tagged version otherwise; etc.), or always issue a version conflict error. Definitely always prefer the branch. The goal is to let the developer override the decisions made by upstream library authors. The practical effect that was already very visible in the existing ecosystem was that packages tended to separate into two clusters. One cluster that used branch dependencies and another that used version dependencies. Both incompatible with each other. It also meant that a lot of people didn't bother to make any version tags at all, which is understandable, but also a very bad influence to the stability of the ecosystem. Semantic versioning, if used correctly, is a very important factor in that regard. Yes, but that's a social problem. We can nudge people in the right direction by printing a warning message, or even by rejecting it completely on code.dlang.org. The compromise solution to allow branch based dependencies only in the version selection file doesn't really have drawbacks, but dodges all of those issues by moving the decision to the root of the dependency tree. I does have obvious drawbacks: If dub.selections.json contains non-exact versions, different developers can get slightly different dependencies. dub.selections.json must fully determine the versions that are going to be used, otherwise it has failed its purpose. You can also check out certain branches locally and use dub add-local or dub add-path to let DUB use it instead of the publicly registered one. This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task. Using dub.selections.json should be the right tool then. As explained above, it isn't. You could of course also use things like git submodules + path based dependencies in this case, there is no need to force everything into one schema (especially when it doesn't fit well, such as versions and branches). Yes, and I'm sure I can come up with even more workarounds ;-)
Re: Make dub part of the standard dmd distribution
On 6/5/15 5:07 AM, Jacob Carlborg wrote: On 2015-06-04 19:44, Steven Schveighoffer wrote: The use case I'm thinking of is basically: import somepackage.somemodule; ... compile - can't find somepackage.somemodule. did you mean somepackage.somemodule from libgeneric on code.dlang.org? (url) to add: dub depend add libgeneric dub depend add libgeneric = found it, added it, added dependency I don't have to look up anything on code.dlang.org, I don't have to do anything, dub figures it all out from the description file online. Sounds like a nice feature but how would that work in practice? I mean, there's nothing that connects a particular D module or package to a particular Dub package. The source of the package does. code.dlang.org can create this automatically. -Steve
Re: Make dub part of the standard dmd distribution
On 01/06/2015 04:48, Manu via Digitalmars-d wrote: Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff (I saw it precedented a few times before, but it doesn't seem that great), but I want a standard place that stuff bundled by linux package managers can agree on. If you use DUB to build, isn't this transparent anyways? -- Bruno Medeiros https://twitter.com/brunodomedeiros
Re: Make dub part of the standard dmd distribution
On 06/01/2015 02:20 AM, Nick Sabalausky wrote: On 05/31/2015 11:48 PM, Manu via Digitalmars-d wrote: As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find. Ditto. Dub's great if you let it be your buildsystem, but as soon as you want to use it as a package-manager-only it becomes an uphill battle with dub fighting back every step of the way (I speak from experience). That problem is worse if your project is a library that you want fetchable through dub. That's why I've been working on some PRs to fix all that. The first step is here (building/expanding on a flag another contributer recently added): https://github.com/D-Programming-Language/dub/pull/572 (Although I think I may need to change --data=libs and separate system libs from dependency libs since linkers expect those to be passed in differently.) The next step will be to add a flag to have that data get output in a compiler-ready argument list (your choice of compiler, of course). That'll let you do something comparable to pkg-config --cflags --libs. All the above is now included in the PR: https://github.com/D-Programming-Language/dub/pull/572 Working example: -- $ cd /home/nick/proj/dub $ dub describe --data=main-source-file --data=options \ --data=versions --data=import-paths '/home/nick/proj/dub/source/app.d' -debug -g -w -version=DubUseCurl -version=Have_dub '-I/home/nick/proj/dub/source/' $ dub describe --compiler=ldc --data=options --data=versions -d-debug -g -w -oq -od=.dub/obj -oq -od=.dub/obj -d-version=DubUseCurl -d-version=Have_dub $ dub describe --data-list --data=options --data=versions debugMode debugInfo warningsAsErrors DubUseCurl Have_dub -- Of course, dub describe --help for the full list of options. Only thing not included yet is this: Then finally, a shortcut flag so you can just do something kinda like: rdmd --build-only `dub describe --cmdargs=rdmd` src/main.d And all your dependencies will be properly referenced (-I..., linker flags, etc).
Re: Make dub part of the standard dmd distribution
On 06/05/2015 02:56 PM, Bruno Medeiros wrote: On 01/06/2015 04:48, Manu via Digitalmars-d wrote: Please declare a standard unix location for D 'includes'. Nobody agrees where in the filesystem D files should be. I use /usr/include/d2/ for my stuff (I saw it precedented a few times before, but it doesn't seem that great), but I want a standard place that stuff bundled by linux package managers can agree on. If you use DUB to build, isn't this transparent anyways? Yes, but dub's buildsystem isn't always suitable for all projects.
Re: Make dub part of the standard dmd distribution
On 06/05/2015 12:45 PM, Nick Sabalausky wrote: https://github.com/D-Programming-Language/dub/pull/572 Working example: -- $ cd /home/nick/proj/dub $ dub describe --data=main-source-file --data=options \ --data=versions --data=import-paths '/home/nick/proj/dub/source/app.d' -debug -g -w -version=DubUseCurl -version=Have_dub '-I/home/nick/proj/dub/source/' $ dub describe --compiler=ldc --data=options --data=versions -d-debug -g -w -oq -od=.dub/obj -oq -od=.dub/obj -d-version=DubUseCurl -d-version=Have_dub $ dub describe --data-list --data=options --data=versions debugMode debugInfo warningsAsErrors DubUseCurl Have_dub -- It also works even if your project doesn't have a dub.json :) $ mkdir ~/my-proj $ cd ~/my-proj $ dub fetch vibe-d $ dub describe vibe-d --data=versions --data=import-paths -version=VibeLibeventDriver -version=Have_vibe_d -version=Have_libevent -version=Have_openssl '-I/home/nick/.dub/packages/vibe-d-0.7.23-beta.1/source/' '-I/home/nick/.dub/packages/libevent-2.0.1_2.0.16/' '-I/home/nick/.dub/packages/openssl-1.1.4_1.0.1g/'
Re: Make dub part of the standard dmd distribution
On Thursday, 4 June 2015 at 15:39:02 UTC, Sönke Ludwig wrote: Am 04.06.2015 um 17:08 schrieb Nick Sabalausky: On 06/04/2015 04:43 AM, Sönke Ludwig wrote: (...) except for making the API stable and supporting the improved build description language. Is there anything you need from me on the latter? (If a particular matter quiets down or I get distracted, I'm liable to forget about it.) I think Dicebot did some work recently (or intended to do?). It's actually not that much work overall. I could also pick up what's there and finalize things over the next days. I got it to the point where I could build simple hello-world using dub.sdl description but haven't worked on it since then. This issue is neither interesting nor important to me, thus finding motivation is rather hard. Dump of last sources can be found here : https://github.com/Dicebot/dub/tree/sdl
Re: Make dub part of the standard dmd distribution
On 6/4/15 11:07 AM, Sönke Ludwig wrote: Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer: On 6/4/15 4:58 AM, Sönke Ludwig wrote: As of recently, you can also directly specify dependencies to the dub init command, for example dub init myproject tango derelict-gl. That's nice. But it still isn't self-explanatory. Nor additive. A command to add/remove dependencies might be useful. But the question is where to stop. Does it make sense to have a command to edit the description? Or add compiler flags? Or manage configurations or sub packages? It also gets a little hairy w.r.t. keeping possible user formatting in the package description file, especially once support for comments gets added. Editing strings that don't affect the build doesn't seem like a necessary thing. An interactive init would probably solve that. However, things like adding or removing a dependency would be very nice without having to know the format of the file. I'm trying to think of things that a simple first-time user would use dub for, aside from adding dependencies. Perhaps creating a debug build. Other than that, I think editing the file for nitty gritty details is probably OK. The use case I'm thinking of is basically: import somepackage.somemodule; ... compile - can't find somepackage.somemodule. did you mean somepackage.somemodule from libgeneric on code.dlang.org? (url) to add: dub depend add libgeneric dub depend add libgeneric = found it, added it, added dependency I don't have to look up anything on code.dlang.org, I don't have to do anything, dub figures it all out from the description file online. I think of things like linux shell on my linuxmint installation (ubuntu based), which uses some cool things: user blah No command 'blah' found, did you mean: Command 'blam' from package 'blam' (universe) Of course, you could just say well, edit the json file!. You really need a format that supports comments... Agreed, that is really needed. It's almost on top of the TODO list. That is good! In any case, this doesn't negate the concerns others have raised. I don't mean to bash dub, it's better than nothing. But I haven't used it for any real project yet. And when I did use it, it was not a straightforward experience. The important part is to not stop with voicing criticism, but to actively help to improve the situation. Opening tickets or voting on existing ones, or even making an enhancement proposal or an actual PR would be the most constructive. Not using dub frequently means I have a very sparse relationship with it. It's kind of that chicken and egg thing. I'm sure if I started having to use dub for a project, I would become more involved :) Sorry for the slightly OT reply, most of this isn't targeted at you, I just went through the newsgroup posts for the first time after a while and needed to get this out. No problem, please keep up the dialogue! -Steve
Re: Make dub part of the standard dmd distribution
Am 04.06.2015 um 19:55 schrieb Dicebot: I got it to the point where I could build simple hello-world using dub.sdl description but haven't worked on it since then. This issue is neither interesting nor important to me, thus finding motivation is rather hard. Dump of last sources can be found here : https://github.com/Dicebot/dub/tree/sdl I'll pick it up from there in a few days.
Re: Make dub part of the standard dmd distribution
On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote: Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer: This would be better, but I still think a way to add dependencies should be supported on the command line. Something like dub depend someproject. You don't always know what a project will need at the beginning. imho if a program consumes a particular configuration file, it should not also write it. Counter examples: Visual Studio projects, MFC, IRC bouncers, and other technologies from Hell. On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote: Sorry for the slightly OT reply, most of this isn't targeted at you, I just went through the newsgroup posts for the first time after a while and needed to get this out. Be sure your work is appreciated, I couldn't imagine working without DUB everyday. As always the NG isn't the masses but some vocal subset of D users. Personally I won't use any library which is not on the DUB registry, learning how to build something is disposable knowledge.
Re: Make dub part of the standard dmd distribution
On a related note, it's also a pity that Reggae mixes incremental build improvements (from which DUB itself could greatly profit, too - just as a Ninja generator for DUB would be a nice feature) with a separate, layered build description. I mean there is of course no reason to not have alternative approaches for build descriptions available in general, but when mixed with a public package repository, it just leads to fragmentation. I can add a ninja generator to dub if you want, with the default being per-package compilation (since right now for dub it's one file or everything at once). But I wrote reggae because: 1. Declarative is preferable but imperative is needed. My (our?) favourite imperative language is D, so I want to specify builds in it. 2. I want a build tool that builds on dub but doesn't require it 3. I wanted it to make it easy to link with C and C++ As I've mentioned before, I know the kind of things I'd want to do with the build system if I had a large and complicated enough project, and I know I wouldn't be able to do it easily using dub alone. As I've also mentioned before, building with dub is just fine for most people. I don't know if fragmentation would be an issue. The packages are still dub packages and I for one will use dub.json/sdl to list my dependencies even if reggae is actually generating the build. Atila
Re: Make dub part of the standard dmd distribution
Am 04.06.2015 um 17:03 schrieb Nick Sabalausky: On 06/04/2015 04:39 AM, Sönke Ludwig wrote: Of course you could make the system completely build tool agnostic, but then you'd basically lose interoperability between packages, as each package might choose its own build tool. With this PR and the upcoming follow-up to add --format=(list|some_compiler|whatever), there is NO issue with package interoperability (as long as a package's import-paths is correct): https://github.com/D-Programming-Language/dub/pull/572 Just to get this straight: Supporting external build tools like that and by writing the proper generator has always been the goal of DUB (being build tool agnostic). The built-in builder is basically mainly there for convenience (although there is no reason not to improve it to the point where it is a worthy competitor). But you still use DUB's build description in that case. What I really meant (and didn't really say) was if you completely remove all build description related fields from dub.json - then you lose interoperability. Note that even WITHOUT dub, packages from different buildsystems *already* interop just fine. Even in C-land. All you ever need for packages to work together is the appropriate build this package command and the import/lib paths for the compiler/linker. Buildsystem is a package-local issue, not a cross-package issue. That's not what I had in mind with interoperability. That would rather be gluing them together manually. Of course you can always do that, but then you lose a lot of potential for avoiding useless/redundant work. It means you are doing the job that the tool should (could) do. This also gets especially interesting and annoying when compiler flags (e.g. version identifiers) don't match, or the dependencies require advanced compiler flags. I really think C/C++ is the last thing we should take as our role model when it comes to this topic (apart from C's strict ABI perhaps).
Re: Make dub part of the standard dmd distribution
Am 04.06.2015 um 17:08 schrieb Nick Sabalausky: On 06/04/2015 04:43 AM, Sönke Ludwig wrote: (...) except for making the API stable and supporting the improved build description language. Is there anything you need from me on the latter? (If a particular matter quiets down or I get distracted, I'm liable to forget about it.) I think Dicebot did some work recently (or intended to do?). It's actually not that much work overall. I could also pick up what's there and finalize things over the next days.
Re: Make dub part of the standard dmd distribution
On 06/04/2015 11:33 AM, Sönke Ludwig wrote: Am 04.06.2015 um 17:03 schrieb Nick Sabalausky: On 06/04/2015 04:39 AM, Sönke Ludwig wrote: Of course you could make the system completely build tool agnostic, but then you'd basically lose interoperability between packages, as each package might choose its own build tool. With this PR and the upcoming follow-up to add --format=(list|some_compiler|whatever), there is NO issue with package interoperability (as long as a package's import-paths is correct): https://github.com/D-Programming-Language/dub/pull/572 Just to get this straight: Supporting external build tools like that and by writing the proper generator has always been the goal of DUB (being build tool agnostic). The built-in builder is basically mainly there for convenience (although there is no reason not to improve it to the point where it is a worthy competitor). But you still use DUB's build description in that case. What I really meant (and didn't really say) was if you completely remove all build description related fields from dub.json - then you lose interoperability. Oh, right. I have no issue with dub *having* a built-in buildsystem. (And it is fine and convenient for a lot of things, even if not everything.) I've just hit roadblocks trying to use other buildsystems instead and am only just now getting around to addressing those issues.
Re: Make dub part of the standard dmd distribution
On 06/04/2015 04:39 AM, Sönke Ludwig wrote: Of course you could make the system completely build tool agnostic, but then you'd basically lose interoperability between packages, as each package might choose its own build tool. With this PR and the upcoming follow-up to add --format=(list|some_compiler|whatever), there is NO issue with package interoperability (as long as a package's import-paths is correct): https://github.com/D-Programming-Language/dub/pull/572 Note that even WITHOUT dub, packages from different buildsystems *already* interop just fine. Even in C-land. All you ever need for packages to work together is the appropriate build this package command and the import/lib paths for the compiler/linker. Buildsystem is a package-local issue, not a cross-package issue.
Re: Make dub part of the standard dmd distribution
Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer: On 6/4/15 4:58 AM, Sönke Ludwig wrote: As of recently, you can also directly specify dependencies to the dub init command, for example dub init myproject tango derelict-gl. That's nice. But it still isn't self-explanatory. Nor additive. A command to add/remove dependencies might be useful. But the question is where to stop. Does it make sense to have a command to edit the description? Or add compiler flags? Or manage configurations or sub packages? It also gets a little hairy w.r.t. keeping possible user formatting in the package description file, especially once support for comments gets added. There was also the suggestion to make dub init (without arguments) interactive, which would make this a really easy matter. This would be better, but I still think a way to add dependencies should be supported on the command line. Something like dub depend someproject. You don't always know what a project will need at the beginning. Of course, you could just say well, edit the json file!. You really need a format that supports comments... Agreed, that is really needed. It's almost on top of the TODO list. In any case, this doesn't negate the concerns others have raised. I don't mean to bash dub, it's better than nothing. But I haven't used it for any real project yet. And when I did use it, it was not a straightforward experience. The important part is to not stop with voicing criticism, but to actively help to improve the situation. Opening tickets or voting on existing ones, or even making an enhancement proposal or an actual PR would be the most constructive. It's a little sad how some people seem to perceive DUB as being static and unable to evolve. That is not the case. It has started out with a small core functionality to get something that benefits a lot of people right away. But everything has been done in a way that allows to extend or generalize things later. Not everyone will agree with its philosophy to provide high level declarative abstractions instead of a low level system that can be used to create abstractions within the build language, but I'm very positive that this approach is much more useful for the vast majority of users (and especially for newcomers). Retrofitting existing build approaches can of course sometimes come with friction, but even that should be solvable one way or another in most cases. In any case, probably all of the mentioned criticism so far seems to be based on nothing more than missing functionality or bugs and nothing that would go against the existing design. An exception is support for other high level tasks, such as deployment targets or similar. This has never been the scope and I don't think we should go that way. On a related note, it's also a pity that Reggae mixes incremental build improvements (from which DUB itself could greatly profit, too - just as a Ninja generator for DUB would be a nice feature) with a separate, layered build description. I mean there is of course no reason to not have alternative approaches for build descriptions available in general, but when mixed with a public package repository, it just leads to fragmentation. Sorry for the slightly OT reply, most of this isn't targeted at you, I just went through the newsgroup posts for the first time after a while and needed to get this out.
Re: Make dub part of the standard dmd distribution
On 06/04/2015 04:43 AM, Sönke Ludwig wrote: Am 01.06.2015 um 08:35 schrieb Nick Sabalausky: On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote: Let's make this part of 2.068: https://issues.dlang.org/show_bug.cgi?id=14636 It's preapproved. Who would want to work on it? IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0. We have collected some things that would be good to have done. There are no big issues left, though, Cool. except for making the API stable and supporting the improved build description language. Is there anything you need from me on the latter? (If a particular matter quiets down or I get distracted, I'm liable to forget about it.)
Re: Make dub part of the standard dmd distribution
Am 04.06.2015 um 12:30 schrieb Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net: On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote: Am 01.06.2015 um 17:16 schrieb Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net: Since this thread seems to have turned into a wishlist for dub features, I'm going to add one: Undeprecate dependencies on git branches. They are really not different from dependencies with fuzzy ~ versions. For both, the exact selected version can be stored in dub.selections.json. Use case: Making a fix to a third-party project and sharing that fix with other team members. Having to create a temporary project on code.dlang.org just for that purpose is both a waste of time for developers and a waste of resources of community infrastructure. They are only deprecated within dub.json. You can still use them within dub.selections.json. This is exactly the wrong way round. dub.selections.json must only contain exact versions. That's its purpose after all. A branch *is* an exact version as far as DUB goes (just one that changes over time). But to make use of any commit relationships within a branch, it would have to gain specific knowledge of GIT or other version management systems. Adding support for commit hashes in dub.selections.json is still TBD, but would be another alternative. BTW, my primary concern with all of these things is retaining a healthy public package eco system. I personally suspect that this is much more important for the language as a whole rather than pleasing every single taste. This doesn't mean that the latter isn't a worthwhile goal, but to me the former has always been more important. What is missing is a proposal how to solve both issues at the same time. See Ruby's bundler as an example: http://bundler.io/v1.9/rationale.html Especially the parts about Gemfile.lock, which is their equivalent of dub.selections.json. But really, branch based dependencies are simply a broken feature. They are completely different to version ranges, because they are not at all comparable (to other branches or versions) - at least without completely committing to GIT and starting to walk the GIT commit graph. Why would you even want to compare them, except for equality? (...) For example for conflict resolution. This is the foremost reason why the *have* to be banned from dub.json. You can't really know if a branch matches/is compatible with any particular version (range) or another branch, so you have to either make an arbitrary guess (always prefer the branch; always prefer the version; always prefer master, but a tagged version otherwise; etc.), or always issue a version conflict error. The practical effect that was already very visible in the existing ecosystem was that packages tended to separate into two clusters. One cluster that used branch dependencies and another that used version dependencies. Both incompatible with each other. It also meant that a lot of people didn't bother to make any version tags at all, which is understandable, but also a very bad influence to the stability of the ecosystem. Semantic versioning, if used correctly, is a very important factor in that regard. The compromise solution to allow branch based dependencies only in the version selection file doesn't really have drawbacks, but dodges all of those issues by moving the decision to the root of the dependency tree. You can also check out certain branches locally and use dub add-local or dub add-path to let DUB use it instead of the publicly registered one. This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task. Using dub.selections.json should be the right tool then. You could of course also use things like git submodules + path based dependencies in this case, there is no need to force everything into one schema (especially when it doesn't fit well, such as versions and branches).
Re: Make dub part of the standard dmd distribution
On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote: Am 01.06.2015 um 17:16 schrieb Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net: Since this thread seems to have turned into a wishlist for dub features, I'm going to add one: Undeprecate dependencies on git branches. They are really not different from dependencies with fuzzy ~ versions. For both, the exact selected version can be stored in dub.selections.json. Use case: Making a fix to a third-party project and sharing that fix with other team members. Having to create a temporary project on code.dlang.org just for that purpose is both a waste of time for developers and a waste of resources of community infrastructure. They are only deprecated within dub.json. You can still use them within dub.selections.json. This is exactly the wrong way round. dub.selections.json must only contain exact versions. That's its purpose after all. See Ruby's bundler as an example: http://bundler.io/v1.9/rationale.html Especially the parts about Gemfile.lock, which is their equivalent of dub.selections.json. But really, branch based dependencies are simply a broken feature. They are completely different to version ranges, because they are not at all comparable (to other branches or versions) - at least without completely committing to GIT and starting to walk the GIT commit graph. Why would you even want to compare them, except for equality? There are only the following cases AFAICS, none of which require an order: 1) Package is not yet in dub.selections.json: = Clone/update repo, checkout the branch, and put the exact commit id into dub.selections.json. 2) Package is already included in dub.selections.json: = All fine, nothing needs to be done. 3) An upgrade of the package has been requested: = Same as 1) There could be situations during dependency resolution where you want to check whether a given commit is part of a branch, but that's a different (and easier) problem. You can also check out certain branches locally and use dub add-local or dub add-path to let DUB use it instead of the publicly registered one. This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task.
Re: Make dub part of the standard dmd distribution
Am 01.06.2015 um 09:19 schrieb Iain Buclaw via Digitalmars-d: On 1 Jun 2015 09:09, Manu via Digitalmars-d digitalmars-d@puremagic.com mailto:digitalmars-d@puremagic.com wrote: On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com mailto:digitalmars-d@puremagic.com wrote: On 1 Jun 2015 08:45, Nick Sabalausky via Digitalmars-d digitalmars-d@puremagic.com mailto:digitalmars-d@puremagic.com wrote: On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org http://dlang.org, and then everyone will agree and fall in line :) Not sure how serious/joking you are about that, but when has declaring a standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config. Leave it to the distribution is the safe option in my experience. To have something along the lines of what Manu wants, I guess we need something like virtualenv to allow building in a clean/standard environment. Yeah, I think I can see 2 parallel problems here. 1. There is a lib installed from a -dev package managed by the distribution... I just want the complementary .d files. (this is what I actually care about) 2. There is some open-source D code which isn't distributed as a binary, it's just a git repo and you fetch it and build it locally. (I find that I rarely need this, so I don't have much opinion on this case) For case 1, my preference would be a distro managed package alongside the lib itself, and installed into a standard location. If dub could pull the bindings I want and put them in some common location, fine. For case 2... I dunno. What if you offer a lib that falls into case 2; source is available, user can fetch and build against it locally, but it contains C code too? dub isn't exactly a build system which can express a complex build environment. I can't create a dub package for my engine, which might be of interest to D gamedevs. In one sense this can be solved at the distribution level. If dub was provided through your package manager, the package maintainers can ensure that dub was configured to understand where all system sources are located (or will be located). Can all of dubs default settings be dumped to json and be overriden? It would be possible to put a .dub/settings.json with custom settings into the distribution package (hmm, /etc/dub/settings.json or similar should probably supported in addition to that...).
Re: Make dub part of the standard dmd distribution
Am 01.06.2015 um 16:49 schrieb Jonathan M Davis: On Monday, 1 June 2015 at 06:20:18 UTC, Nick Sabalausky wrote: Ditto. Dub's great if you let it be your buildsystem, but as soon as you want to use it as a package-manager-only it becomes an uphill battle with dub fighting back every step of the way (I speak from experience). That problem is worse if your project is a library that you want fetchable through dub. And this is when Jacob Carlborg chimes in and says I told you so. ;) His favorite complaint about dub has always been that it combined package management and the build tool into one. This is actually completely irrelevant to the issue. Making the build part separate would not at all solve this. Any work on such a separate build tool (e.g. to support other languages) could instead as well go into DUB itself. Of course you could make the system completely build tool agnostic, but then you'd basically lose interoperability between packages, as each package might choose its own build tool. dub works fantastically if you want to do everything in the standard way without any funny stuff, but it clearly doesn't have the power of a tool like make, cmake, etc. As soon as you need to do anything funny - like include anything related to other languages in your build, or put things in a specific layout because of some company-specific thing, or anything that wasn't explicitly planned for by the folks writing dub, dub just won't work. This is really not true. There is currently just one limitation w.r.t. the directory structure and that is the naming scheme of the folder to which packages are downloaded (somepackage-1.0.0). There are some packages that put their sources into the root of the repository and expect to be put into a folder with the name of the repository, which will then be added as an import folder. We are going to solve that. Everything else can be done with the sourcePaths, sourceFiles, importPaths, ... settings. You basically have the same possibilities as you have when directly invoking the compiler. Separate language support is still missing, but you always have the possibility to add generic preBuildCommands or preGenerateCommands to invoke separate tools or scripts. Of course it's desirable to get everything done with the same tool instead of relying on additional build time dependencies, but it's still *possible* to do these things. Now, to be fair, I don't know how you can be pulling in all kinds of stray libraries with who-knows-what for their build systems and expect it to work very well via a single tool (at minimum, you have the problem that the machine that you're trying to build them on will likely be missing some of those tools), but that's pretty much what we need to handle the general case. Without that, dub will work great for the normal case (which _will_ be plenty for many, many projects), but it won't work in the general case - and corporate folks in particular are probably going to have to skip out on using it because of abnormal requirements on their part. Even simply splitting dub out so that it can pull in packages without necessarily being able to build them would be nice (maybe adding a flag to the dub.json file indicating whether dub can be used as a build tool for the project or whether it's supposed to just act as a package manager for it), though that does add the problem of code.dlang.org becoming more confusing, since you wouldn't be able to rely on all projects on it being built for you by dub as part of the process of grabbing packages. I guess using the proper targetType and preBuildCommand settings it should already be possible to invoke a separate build tool and still integrate the generated object/library files into the rest of the DUB build process. But I'd have to try that out - it would definitely be good to have something like this added to the cookbook [1]. Of course things like these always potentially degrade interoperability. DUB's build description was chosen explicitly to avoid unnecessary dependencies on a particular compiler/platform, which is where the most general build tools (especially make) utterly fail. It's a complicate problem, and dub went a route which works in the 90% case, but doesn't work for more complicated cases, which will make dub unusable for some projects - especially corporate ones. I don't know how fixable it is without a major redesign of dub. I don't see anything that wouldn't be fixable. In fact, there are really few things that are not possible now, even if not in the most comfortable way.
Re: Make dub part of the standard dmd distribution
Am 01.06.2015 um 08:35 schrieb Nick Sabalausky: On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote: Let's make this part of 2.068: https://issues.dlang.org/show_bug.cgi?id=14636 It's preapproved. Who would want to work on it? IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0. We have collected some things that would be good to have done. There are no big issues left, though, except for making the API stable and supporting the improved build description language. This should be done within another compiler release cycle.
Re: Make dub part of the standard dmd distribution
Am 01.06.2015 um 17:16 schrieb Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net: Since this thread seems to have turned into a wishlist for dub features, I'm going to add one: Undeprecate dependencies on git branches. They are really not different from dependencies with fuzzy ~ versions. For both, the exact selected version can be stored in dub.selections.json. Use case: Making a fix to a third-party project and sharing that fix with other team members. Having to create a temporary project on code.dlang.org just for that purpose is both a waste of time for developers and a waste of resources of community infrastructure. They are only deprecated within dub.json. You can still use them within dub.selections.json. But really, branch based dependencies are simply a broken feature. They are completely different to version ranges, because they are not at all comparable (to other branches or versions) - at least without completely committing to GIT and starting to walk the GIT commit graph. You can also check out certain branches locally and use dub add-local or dub add-path to let DUB use it instead of the publicly registered one.
Re: Make dub part of the standard dmd distribution
Am 01.06.2015 um 20:49 schrieb Steven Schveighoffer:
(...)
Sorry, I suppose it's a small burden if you use dub frequently. I don't.
Look what happens when I initialize a dub project:
{
name: testdub,
description: A minimal D application.,
copyright: Copyright © 2015, steves,
authors: [steves],
dependencies: {
}
}
Cool, I can figure out how to massage all the existing strings, except,
hm... dependencies. What goes in there?
As of recently, you can also directly specify dependencies to the dub
init command, for example dub init myproject tango derelict-gl. There
was also the suggestion to make dub init (without arguments)
interactive, which would make this a really easy matter.
Re: Make dub part of the standard dmd distribution
On 2015-06-04 12:30, Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net wrote: On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote: Am 01.06.2015 um 17:16 schrieb Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net: You can also check out certain branches locally and use dub add-local or dub add-path to let DUB use it instead of the publicly registered one. This is not an option in a larger team. Everyone would have to do that manually, or you'd need to write a custom script to do what is actually the package manager's task. I would prefer to just stick a Git URL + branch in the dub.json file, just as with the Gemfile in Bundler. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 6/4/15 3:49 PM, ponce wrote: On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote: Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer: This would be better, but I still think a way to add dependencies should be supported on the command line. Something like dub depend someproject. You don't always know what a project will need at the beginning. imho if a program consumes a particular configuration file, it should not also write it. Counter examples: Visual Studio projects, MFC, IRC bouncers, and other technologies from Hell. That program is not consuming a configuration file, it's editing it. Call it dub-depend if you want. -Steve
Re: Make dub part of the standard dmd distribution
On 6/4/15 4:58 AM, Sönke Ludwig wrote:
Am 01.06.2015 um 20:49 schrieb Steven Schveighoffer:
(...)
Sorry, I suppose it's a small burden if you use dub frequently. I don't.
Look what happens when I initialize a dub project:
{
name: testdub,
description: A minimal D application.,
copyright: Copyright © 2015, steves,
authors: [steves],
dependencies: {
}
}
Cool, I can figure out how to massage all the existing strings, except,
hm... dependencies. What goes in there?
As of recently, you can also directly specify dependencies to the dub
init command, for example dub init myproject tango derelict-gl.
That's nice. But it still isn't self-explanatory. Nor additive.
There
was also the suggestion to make dub init (without arguments)
interactive, which would make this a really easy matter.
This would be better, but I still think a way to add dependencies should
be supported on the command line. Something like dub depend someproject.
You don't always know what a project will need at the beginning.
Of course, you could just say well, edit the json file!. You really
need a format that supports comments...
In any case, this doesn't negate the concerns others have raised.
I don't mean to bash dub, it's better than nothing. But I haven't used
it for any real project yet. And when I did use it, it was not a
straightforward experience.
-Steve
Re: Make dub part of the standard dmd distribution
On Thursday, 4 June 2015 at 13:07:50 UTC, Steven Schveighoffer wrote: That's nice. But it still isn't self-explanatory. Nor additive. Do you think it'd be better to have a list of flags to specify dependancies? Like : dub init myProj -d tango -d derelict-gl --dependency=vibe-d This would be better, but I still think a way to add dependencies should be supported on the command line. Something like dub depend someproject. You don't always know what a project will need at the beginning. Agree here, this is something I plan on working on soon. Of course, you could just say well, edit the json file!. You really need a format that supports comments... In any case, this doesn't negate the concerns others have raised. I don't mean to bash dub, it's better than nothing. But I haven't used it for any real project yet. And when I did use it, it was not a straightforward experience. -Steve
Re: Make dub part of the standard dmd distribution
On 2015-06-02 10:54, ketmar wrote: that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. i extended CTFE engine with simple file i/o functions and system call with config-defined executables (something like sudoers list) just to see if it will be usable. and it's surprisingly fun even with all the limits and without hooks for module imports and so on. this system can be extended to allow writing arbitrary subcommands (like git). just import the corresponding subcommand module, if any, and CTFE it's invocation point. bingo! the system that can be extended without recompiling the compiler. and user can add subcommands on per-project base! sadly, this seems to get no wow!s. :-( I think it's pretty cool. But I like the approach SDC has taken even more. Using the LLVM JIT to run CTFE code. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2015-06-02 16:52, Steven Schveighoffer wrote: Actually, I don't. dmd *.d generally works fine. I if it works for you, great. It doesn't work for me. And that shell globbing doesn't work on Windows. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2015-06-02 17:01, Nick Sabalausky wrote: Related to this, does anyone happen to recall why rdmd uses dmd -v to get dependencies instead of dmd -deps? IIRC, it used to use -deps back at one time. I would assumed it uses -v because it existed before -deps? I guess -deps was explicitly create because of the rdmd use case. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On Wed, 03 Jun 2015 13:45:06 +0200, Jacob Carlborg wrote: On 2015-06-02 10:54, ketmar wrote: that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. i extended CTFE engine with simple file i/o functions and system call with config-defined executables (something like sudoers list) just to see if it will be usable. and it's surprisingly fun even with all the limits and without hooks for module imports and so on. this system can be extended to allow writing arbitrary subcommands (like git). just import the corresponding subcommand module, if any, and CTFE it's invocation point. bingo! the system that can be extended without recompiling the compiler. and user can add subcommands on per-project base! sadly, this seems to get no wow!s. :-( I think it's pretty cool. But I like the approach SDC has taken even more. Using the LLVM JIT to run CTFE code. so SDC will benefit from JITting, as it's not required to modify CTFE evaluator in any way to implement my idea. just adding some more internal functions (like `import()`), which are nothing more than call `interpret`, use results). porting that entry points to SDC will be trivial. signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Wednesday, 3 June 2015 at 01:23:37 UTC, Manu wrote: I'm inclined to suggest, this would be an excellent focus for 2.068... The compilers need to become aligned, and ABI issues seems like a critical issue to focus on. There are much bigger fish to fry at this point. Phobos breaks ABI compatibility with every single release anyway. - David
Re: Make dub part of the standard dmd distribution
On 2015-06-01 16:49, Jonathan M Davis wrote: And this is when Jacob Carlborg chimes in and says I told you so. ;) His favorite complaint about dub has always been that it combined package management and the build tool into one. :) -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2015-06-01 20:39, Nick Sabalausky wrote: Unlike rdmd, dub always tries to compile all sources regardless of whether they're actually imported. This has been a constant source of problems for me, including breakage of conditional importing under various circumstances. Isn't that the only way to compile it if the project is a library? without having to use workarounds like create a new module which imports all other modules. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2015-06-01 17:16, Marc =?UTF-8?B?U2Now7x0eiI=?= schue...@gmx.net wrote: Since this thread seems to have turned into a wishlist for dub features, I'm going to add one: Undeprecate dependencies on git branches. They are really not different from dependencies with fuzzy ~ versions. For both, the exact selected version can be stored in dub.selections.json. That's a good point, as long as people understand how to use dub.selections.json [1]. But I would rather not allow branches in the registry. I would prefer that a git URL could be added directly to dub.json instead. [1] https://github.com/github/gitignore/pull/1444 -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2015-06-01 20:03, Johannes Pfau wrote: Create a package format: .dlib (simply a renamed .tar.xz) Contents: - PACKAGEINFO //meta information (version) - include/* //include files - doc/* //documentation in standard format for IDEs - lib/dmd/libfoo.a - lib/gdc/libfoo.a Write a small tool xdmd. Implement: //simply move to /usr/share/dlib/libfoo/version/ //or ~/.share/dlib/libfoo/version/ xdmd --install libfoo.dlib Note: when installing using a systems package manager, packages could be installed in the same way: pacman -S libfoo could call xdmd --install. //extract /usr/share/dlib/libfoo/0.9.3/libfoo.dlib //to temporary directory. Pass link and include flags and //rest of arguments to dmd xdmd --library=foo:0.9.3 test.d //directly use local package xdmd --library=libfoo-0.9.3.dlib test.d This is essentially are very slimmed down dub. Would be a nice weekend exercise to write such a tool. So, a package manager ;). I tried that, but people were complaining that I used Ruby for the meta information and that binary packages would cause problems. Then Dub came and I gave up on that project. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On Mon, 01 Jun 2015 11:59:12 +, extrawurst wrote: dub as a build tool sux. no need to discuss that, it simply sux. why do you think it sucks ? 'cause it's not more than a simple script in it's core. it can't do good conditional builds, it can't track dependencies for generated files or non-D code, and so on. it's rdmd with some bells and some features stripped. this is fine for simple pure D projects, but for complex projects one have to use supplement build tool. and if we have *two* build tools in use, it's logical to remove one that is more tied to the language. and it's dub. so while it's ok to use dub to build simple things, it's not a good build system. my k8jam, while can't do consolidated (i.e. non-separate ;-) builds, can build D code with C/C++ libraries (which are built from source too), generate files, track dependencies, do 'configure' work, allows dlang.require operators with dependency tracking between libraries and so on. it can't download libraries, though -- i choose to not implement that, hoping that dub can fulfill that role. signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Mon, 01 Jun 2015 13:11:42 +, Adam D. Ruppe wrote: One of the things I really like about D is how modules are encapsulated into individual files. The code is there, the documentation is there, the tests are there. No separate headers or anything else. i like this too. But even if I don't use it myself, it would be kinda nice if the repository could handle my files just so I can toss it up there and not have to answer questions about that anymore. I'd be willing to add them all individually to the repo, even listing the dependencies manually if I had to (tip: make this a web form that I can just submit to add stuff to it, no json upload) but I'm not willing to separate my little repo into 25 other repos and separate my folder into 125 other folders to be added to it. yep. i don't feel like splitting my IV repo to alot of individual repositories. quite the contrary, i'm moving modules from separate repositories to IV when they are ready. some of IV things are perfectly usable with vanilla, so they can go to code.dlang.org, but... but there is no way to do that now. signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote: A protocol to be used between dmd and dub (or any other package fetcher) would be a good first step. that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. i extended CTFE engine with simple file i/o functions and system call with config-defined executables (something like sudoers list) just to see if it will be usable. and it's surprisingly fun even with all the limits and without hooks for module imports and so on. this system can be extended to allow writing arbitrary subcommands (like git). just import the corresponding subcommand module, if any, and CTFE it's invocation point. bingo! the system that can be extended without recompiling the compiler. and user can add subcommands on per-project base! sadly, this seems to get no wow!s. :-( signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote: On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote: A protocol to be used between dmd and dub (or any other package fetcher) would be a good first step. that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. https://github.com/atilaneves/reggae Atila
Re: Make dub part of the standard dmd distribution
On 2015-06-01 20:49, Steven Schveighoffer wrote: I'm not defending DIP11, just that the fact that it could be done without requiring an extra dependencies file. One usually needs a file to set all compiler and linker flags and other kinds of configurations. Dub allows you to do this as well in cross-platform way. So when you already have all that, just put the dependencies there and be done with it. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2/06/2015 8:54 p.m., ketmar wrote: On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote: A protocol to be used between dmd and dub (or any other package fetcher) would be a good first step. that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. i extended CTFE engine with simple file i/o functions and system call with config-defined executables (something like sudoers list) just to see if it will be usable. and it's surprisingly fun even with all the limits and without hooks for module imports and so on. this system can be extended to allow writing arbitrary subcommands (like git). just import the corresponding subcommand module, if any, and CTFE it's invocation point. bingo! the system that can be extended without recompiling the compiler. and user can add subcommands on per-project base! sadly, this seems to get no wow!s. :-( I say wow. Okay not really I know it is possible and not all the hard. Personally I think it is a rather an awesome possibility. Really what I want is to add the ability to add on the compiler side symbols that can be CTFE'd. Registered at start of runtime. Add on shared library support and wamo. You can add stuff like this pretty arbitrary :)
Re: Make dub part of the standard dmd distribution
On 2015-06-01 18:56, Dicebot wrote: Though it is hard to invent a package management for interpreted language that is not insane. It is much better with native ones because runtime dependencies are separated from development dependencies. Rubygems supports development dependencies. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On Tue, 02 Jun 2015 09:15:12 +, Atila Neves wrote: On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote: On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote: A protocol to be used between dmd and dub (or any other package fetcher) would be a good first step. that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. https://github.com/atilaneves/reggae Atila hm. looks interesting, albeit a little too noisy for my taste. signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Tue, 02 Jun 2015 21:12:14 +1200, Rikki Cattermole wrote: I say wow. Okay not really I know it is possible and not all the hard. Personally I think it is a rather an awesome possibility. adding some CTFE functions is not that hard at all. it's not documented, but one can look at import() and pragma() to see how they're doing their things and do the same. that's what i did, and it took me one day to write a working PoC. i promised to make a series of articles on that for TWiD, but still didn't done that. mea maxima culpa. Really what I want is to add the ability to add on the compiler side symbols that can be CTFE'd. Registered at start of runtime. i'm afraid i didn't get it. can you provide a sample? Add on shared library support and wamo. You can add stuff like this pretty arbitrary :) you mean adding symbols with plugin-like system? i'm not sure that it will be easy. anyway, with well-defined CTFE API to access file system and execute commands (with restrictions, of course, so D programs will not go wild doing rm -rf ~/ ;-) one can write modules that contains functions that acts like filters, executing external binaries and massaging input/output. with some hooks added (like call this CTFE function (if it present) when module importing fails) and ability to add -I/-J pathes in that hooks (restricted to some library root dir) one can write CTFE rdmd with automatic package downloading. i think i have to make a draft and PoC for that, so people can try it and see how k00l it is. ;-) signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Tuesday, 2 June 2015 at 07:02:50 UTC, Jacob Carlborg wrote: On 2015-06-01 18:56, Dicebot wrote: Though it is hard to invent a package management for interpreted language that is not insane. It is much better with native ones because runtime dependencies are separated from development dependencies. Rubygems supports development dependencies. You must have though of different development dependencies. I don't mean tools and stuff like that but all sorts of small static libraries (in native compilation world). With scripting language this unavoidably propagates on the end user unless you merge all deps into your sources inline for distribution (which no one does).
Re: Make dub part of the standard dmd distribution
On Tuesday, 2 June 2015 at 09:15:13 UTC, Atila Neves wrote: https://github.com/atilaneves/reggae Atila If this works up to the expectation, I will only need to write a simple code.dlang.org package fetcher and will never need to use dub again :)
Re: Make dub part of the standard dmd distribution
On Tuesday, 2 June 2015 at 18:56:22 UTC, Dicebot wrote: On Tuesday, 2 June 2015 at 09:15:13 UTC, Atila Neves wrote: https://github.com/atilaneves/reggae Atila If this works up to the expectation, I will only need to write a simple code.dlang.org package fetcher and will never need to use dub again :) Destroy away! If it doesn't work like you like, let me know. Atila
Re: Make dub part of the standard dmd distribution
On 2015-06-02 20:54, Dicebot wrote: You must have though of different development dependencies. I don't mean tools and stuff like that but all sorts of small static libraries (in native compilation world). With scripting language this unavoidably propagates on the end user unless you merge all deps into your sources inline for distribution (which no one does). Oh, you mean like that. -- /Jacob Carlborg
Re: Make dub part of the standard dmd distribution
On 2 June 2015 at 04:03, Johannes Pfau via Digitalmars-d digitalmars-d@puremagic.com wrote: Am Mon, 1 Jun 2015 15:26:51 +1000 schrieb Manu via Digitalmars-d digitalmars-d@puremagic.com: So, DMD/LDC/GDC know where to look to find these packages? IIRC no. What happens if the package includes a binary lib? That that, I still want someone to declare an official path for D 'includes' in the *nix filesystem, so D lib packages have somewhere to install... GDC looks in /usr/include/d2. Dicebot changed that to /usr/include/d/gdc for archlinux and there are some reasons why this is (for now) necessary: Sharing D sources/headers between compilers is possible, but there are some headers (druntime,phobos) which can be compiler specific. As long as these are not in a common import path it's not a problem though. The main problem is we don't have ABI compatibility. This means we can't share the libraries between compilers. So we could make 'import foo' work but linking with -lfoo without manually adjusting linker paths is not possible. There are some solutions, all have drawbacks: * Have compiler specific directories for libraries (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for static libraries. Can only have one version of a library installed * have per-library directories. I guess dub uses this. Can have multiple versions of the same library. Drawback: compiler can't know the correct library path.[1] * Shared libraries should be installed in a common location (/usr/lib) anyway. This is only possible if we have ABI compatibility[2]. Library versioning is limited to the standard C/C++ like versioning. I guess the main reason why we can't have a C/C++ like out of the box experience is ABI compatibility. There'll always be some quirks as long as we don't fix this. I'm inclined to suggest, this would be an excellent focus for 2.068... The compilers need to become aligned, and ABI issues seems like a critical issue to focus on.
Re: Make dub part of the standard dmd distribution
2015-06-01 20:03 GMT+02:00 Johannes Pfau via Digitalmars-d digitalmars-d@puremagic.com: The main problem is we don't have ABI compatibility. This means we can't share the libraries between compilers. So we could make 'import foo' work but linking with -lfoo without manually adjusting linker paths is not possible. There are some solutions, all have drawbacks: * Have compiler specific directories for libraries (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for static libraries. Can only have one version of a library installed * have per-library directories. I guess dub uses this. Can have multiple versions of the same library. Drawback: compiler can't know the correct library path.[1] * Shared libraries should be installed in a common location (/usr/lib) anyway. This is only possible if we have ABI compatibility[2]. Library versioning is limited to the standard C/C++ like versioning. I guess the main reason why we can't have a C/C++ like out of the box experience is ABI compatibility. There'll always be some quirks as long as we don't fix this. Glad to see that post ! But in addition of the ABI compatibility between compilers, we would need to have ABI compatibility between releases of the same major version if we were to walk the C[++] route, and we do not want that, for various reason, the most obvious one being that you won't be able to add an attribute like @nogc / nothrow to a free function without doing a major release. There is good thing about C way of doing thing, but there is also some bad. For example, you can't have multiple versions of the same library installed - you don't need it -, but D definitely needs that. That also leads to major releases ending up in a different package - think qt4 / qt5 -. So, while C use SemVer for ABI compatibility, we should use SemVer for source compatibility, or we'll end up with an unmanageable mess.
Re: Make dub part of the standard dmd distribution
On 2/06/2015 10:15 p.m., ketmar wrote:
On Tue, 02 Jun 2015 21:51:54 +1200, Rikki Cattermole wrote:
Essentially its compiler plugins that can add news types +
free-functions as if it was D code provided by source code only also
able to modify e.g. AST directly.
So while in of itself would not provide new language features, it could
add some very nice behavior to existing ones.
So what you want with commands ext. would be done in a shared library.
It have e.g. a struct added into object.d(i).
__CEIFileSystem.delete(./tmp/something);
Of course adding these plugins should be pretty explicit. Package
maintainers for example shouldn't auto install any. There should be no
way to auto install them.
Basically dmd-plugin-dub-bin would be an example package. That would add
dub support directly into dmd.
--
@__CEIDub.Dependency(vibe-d, =0.7.22)
void main() {
// what have you
}
--
Or:
--
static assert(__CEIDub.dependency(vibe.d, =0.7.22), Requires
vibe-d 0.7.22 or newer.);
void main() {
// what have you
}
--
That way it can be used for e.g. static-if and template if.
Of course I'm sure Walter would go nuts at such an idea, if it was
seriously proposed.
So perhaps dmd doesn't support it. Instead a new version of dmd is also
shipped (dmdext) with this enabled ;)
With the community agreeing that we should try to keep as close to dmd
as possible and only reverting to dmdext + plugins to help newbies and
where it just wouldn't be possible to not have an external tool. Which
most likely uses multiple compilation or something else not so nice.
actually, that should be doable with DDMD on non-windows systems*, i
believe. DDMD -- to avoid writing plugins in other languages. ;-) and
with DDMD plugins will have full access to frontend internals, including
AST manipulation.
* non-windows, as dll support on windows is still not here, i believe.
I've been able to do it a little bit already with my Web server. Haven't
hit its limit so far. In other words as long as exceptions aren't used
and its never unloaded, should be ok. Anyway, do you really think we
would get that sort of support for a couple of releases?
the downside of this approach is that it's highly compiler-specific. i.e.
one will force to have different plugins for GDC/LDC/SDC/etc. ;-) and
with pure CTFE API without plugins, only with external executables, it
will be compiler-agnostic, any compiler using DMDFE will get it for free.
only SDC will be forced to rewrite the CTFE part, but other than that all
CTFE D code will work the same.
Yeah it is, but it would also mean better customization of the build
process!
i mean, 'cmon, we have such great built-in scripting engine, let's use it!
Indeed lets. But can we also not go modifying the language? I'm just
suggesting injecting of symbols that execute compiler side actions. Your
suggesting language changes. Even if it is a new pragma.
your samples can be built ontop of my subcommand proposal, for example.
let's say that compiler will try to automatically do static import
DMD.subcommands.dub when programmer writing something like
`DMD.dub.Dependency(vibe-d, =0.7.22);`, falling back to DMD.failure
(name, args) if there is no dub subcommand package.
and then subcommand can execute dub, parse it's output and do what it
wants. or one can make a local override for subcommand -- without even
compiling separate plugin.
that's how i see it.
We really need to toy with these ideas properly and implement each. Then
its just a matter of time to convince the higher ups that it should be
merged.
Re: Make dub part of the standard dmd distribution
On 2/06/2015 9:32 p.m., ketmar wrote:
On Tue, 02 Jun 2015 21:12:14 +1200, Rikki Cattermole wrote:
I say wow.
Okay not really I know it is possible and not all the hard. Personally I
think it is a rather an awesome possibility.
adding some CTFE functions is not that hard at all. it's not documented,
but one can look at import() and pragma() to see how they're doing
their things and do the same. that's what i did, and it took me one day
to write a working PoC.
i promised to make a series of articles on that for TWiD, but still
didn't done that. mea maxima culpa.
Really what I want is to add the ability to add on the compiler side
symbols that can be CTFE'd.
Registered at start of runtime.
i'm afraid i didn't get it. can you provide a sample?
Add on shared library support and wamo. You can add stuff like this
pretty arbitrary :)
you mean adding symbols with plugin-like system? i'm not sure that it
will be easy. anyway, with well-defined CTFE API to access file system
and execute commands (with restrictions, of course, so D programs will
not go wild doing rm -rf ~/ ;-) one can write modules that contains
functions that acts like filters, executing external binaries and
massaging input/output.
with some hooks added (like call this CTFE function (if it present) when
module importing fails) and ability to add -I/-J pathes in that hooks
(restricted to some library root dir) one can write CTFE rdmd with
automatic package downloading.
i think i have to make a draft and PoC for that, so people can try it and
see how k00l it is. ;-)
Essentially its compiler plugins that can add news types +
free-functions as if it was D code provided by source code only also
able to modify e.g. AST directly.
So while in of itself would not provide new language features, it could
add some very nice behavior to existing ones.
So what you want with commands ext. would be done in a shared library.
It have e.g. a struct added into object.d(i).
__CEIFileSystem.delete(./tmp/something);
Of course adding these plugins should be pretty explicit. Package
maintainers for example shouldn't auto install any. There should be no
way to auto install them.
Basically dmd-plugin-dub-bin would be an example package. That would add
dub support directly into dmd.
--
@__CEIDub.Dependency(vibe-d, =0.7.22)
void main() {
// what have you
}
--
Or:
--
static assert(__CEIDub.dependency(vibe.d, =0.7.22), Requires
vibe-d 0.7.22 or newer.);
void main() {
// what have you
}
--
That way it can be used for e.g. static-if and template if.
Of course I'm sure Walter would go nuts at such an idea, if it was
seriously proposed.
So perhaps dmd doesn't support it. Instead a new version of dmd is also
shipped (dmdext) with this enabled ;)
With the community agreeing that we should try to keep as close to dmd
as possible and only reverting to dmdext + plugins to help newbies and
where it just wouldn't be possible to not have an external tool. Which
most likely uses multiple compilation or something else not so nice.
Re: Make dub part of the standard dmd distribution
On Tue, 02 Jun 2015 21:51:54 +1200, Rikki Cattermole wrote:
Essentially its compiler plugins that can add news types +
free-functions as if it was D code provided by source code only also
able to modify e.g. AST directly.
So while in of itself would not provide new language features, it could
add some very nice behavior to existing ones.
So what you want with commands ext. would be done in a shared library.
It have e.g. a struct added into object.d(i).
__CEIFileSystem.delete(./tmp/something);
Of course adding these plugins should be pretty explicit. Package
maintainers for example shouldn't auto install any. There should be no
way to auto install them.
Basically dmd-plugin-dub-bin would be an example package. That would add
dub support directly into dmd.
--
@__CEIDub.Dependency(vibe-d, =0.7.22)
void main() {
// what have you
}
--
Or:
--
static assert(__CEIDub.dependency(vibe.d, =0.7.22), Requires
vibe-d 0.7.22 or newer.);
void main() {
// what have you
}
--
That way it can be used for e.g. static-if and template if.
Of course I'm sure Walter would go nuts at such an idea, if it was
seriously proposed.
So perhaps dmd doesn't support it. Instead a new version of dmd is also
shipped (dmdext) with this enabled ;)
With the community agreeing that we should try to keep as close to dmd
as possible and only reverting to dmdext + plugins to help newbies and
where it just wouldn't be possible to not have an external tool. Which
most likely uses multiple compilation or something else not so nice.
actually, that should be doable with DDMD on non-windows systems*, i
believe. DDMD -- to avoid writing plugins in other languages. ;-) and
with DDMD plugins will have full access to frontend internals, including
AST manipulation.
* non-windows, as dll support on windows is still not here, i believe.
the downside of this approach is that it's highly compiler-specific. i.e.
one will force to have different plugins for GDC/LDC/SDC/etc. ;-) and
with pure CTFE API without plugins, only with external executables, it
will be compiler-agnostic, any compiler using DMDFE will get it for free.
only SDC will be forced to rewrite the CTFE part, but other than that all
CTFE D code will work the same.
i mean, 'cmon, we have such great built-in scripting engine, let's use it!
your samples can be built ontop of my subcommand proposal, for example.
let's say that compiler will try to automatically do static import
DMD.subcommands.dub when programmer writing something like
`DMD.dub.Dependency(vibe-d, =0.7.22);`, falling back to DMD.failure
(name, args) if there is no dub subcommand package.
and then subcommand can execute dub, parse it's output and do what it
wants. or one can make a local override for subcommand -- without even
compiling separate plugin.
that's how i see it.
signature.asc
Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Tuesday, 2 June 2015 at 09:33:24 UTC, ketmar wrote: On Tue, 02 Jun 2015 09:15:12 +, Atila Neves wrote: On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote: On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote: A protocol to be used between dmd and dub (or any other package fetcher) would be a good first step. that was the thing i once proposed. see, we have a powerful scripting language inside DMD: D! yet we never used all it's power to do something really exciting -- like, for example, preparing command lines for external package fetching tool and parsing the answers. instead of providing a simple module for that, it all goes to be hardcoded. https://github.com/atilaneves/reggae Atila hm. looks interesting, albeit a little too noisy for my taste. It depends on what you want to do: the idea is to use high-level utility functions as much as possible, while at the same time offering low-level primitives that those high-level ones are based on. If it can be made simpler, I'd really like to know. Atila
Re: Make dub part of the standard dmd distribution
Well, there's this thread now: http://forum.dlang.org/thread/ranqlmrjornlvopsu...@forum.dlang.org#post-ranqlmrjornlvopsuris:40forum.dlang.org Atila On Tuesday, 2 June 2015 at 13:22:25 UTC, ketmar wrote: On Tue, 02 Jun 2015 12:12:06 +, Atila Neves wrote: https://github.com/atilaneves/reggae Atila hm. looks interesting, albeit a little too noisy for my taste. It depends on what you want to do: the idea is to use high-level utility functions as much as possible, while at the same time offering low-level primitives that those high-level ones are based on. If it can be made simpler, I'd really like to know. not that i have something better to offer. it's readable and looks fine. besides, i'm really sux at design things. ;-)
Re: Make dub part of the standard dmd distribution
On Tue, 02 Jun 2015 13:44:10 +, Wyatt wrote: On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote: sadly, this seems to get no wow!s. :-( If no one knows about it, wow!s will not happen. Make PRs. i mentioned the idea several times in NG, it fades almost without any attention. honestly, i have alot of hobby projects, so before writing such complex things (and it's fairly complex, considering design and documentation) i want to know if there will be enough attention to it. as people seems to pass it by, i have no motivation of doing that work. i have alot of things i want to add to Aliced, so no wow, that would be nice to try! replies means move that down in my TODO list. signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Wed, 03 Jun 2015 01:48:13 +1200, Rikki Cattermole wrote:
Nope, it can be just another function call.
--
auto something(string text);
@something(...)
void another() {
}
--
Where something is specified on the compiler side.
UDA's + static assert imply compile time literals, which is what we
want.
for me UDA syntax that actually calling some function seems
counterintuitive. like well, i know that it's a simple declaration...
WUT?! how it managed to execute the code?!
I think we want something very similar. We just haven't quite merged our
ideas just yet.
yes, seems that it's the same goal, but slightly different approaches. i
think i should do a draft for API and write some explanations first, so
it can be discussed in details. 'cause now i have a foggy picture in my
head which is hard to describe.
signature.asc
Description: PGP signature
Re: Make dub part of the standard dmd distribution
On 3/06/2015 1:33 a.m., ketmar wrote:
On Tue, 02 Jun 2015 22:30:47 +1200, Rikki Cattermole wrote:
* non-windows, as dll support on windows is still not here, i believe.
I've been able to do it a little bit already with my Web server. Haven't
hit its limit so far. In other words as long as exceptions aren't used
and its never unloaded, should be ok. Anyway, do you really think we
would get that sort of support for a couple of releases?
it seems that nobody on windows really needs that. ;-)
Indeed lets. But can we also not go modifying the language? I'm just
suggesting injecting of symbols that execute compiler side actions. Your
suggesting language changes. Even if it is a new pragma.
actually, no language changes at all. CTFE is already here, and it is
used in semantic analysis stage. there are no changes to the language,
besides adding some hooks and defining an API. it's invisible from the
user's POV and doesn't require changing of specs. now CTFE is started
explicitly, by using mixins and template evaluation. with my idea it will
be started implicitly when some condition is met. sure, that conditions
must be documented, but it will not change the language drastically.
Nope, it can be just another function call.
--
auto something(string text);
@something(...)
void another() {
}
--
Where something is specified on the compiler side.
UDA's + static assert imply compile time literals, which is what we want.
Something that is important to note is, these symbols like something
defined by the compiler should never hit the backend. If they are
referenced they should be forced to be called.
We really need to toy with these ideas properly and implement each. Then
its just a matter of time to convince the higher ups that it should be
merged.
yes, i believe that this is the best approach. implement both and let the
best solution win in a honest competition. ;-)
i don't feel very creative right now, but implementing that idea is in my
TODO list, and it's not at the bottom of list. so eventually (month? two?
ten years?) i'll write a PoC.
I think we want something very similar. We just haven't quite merged our
ideas just yet.
Re: Make dub part of the standard dmd distribution
On Tue, 02 Jun 2015 12:12:06 +, Atila Neves wrote: https://github.com/atilaneves/reggae Atila hm. looks interesting, albeit a little too noisy for my taste. It depends on what you want to do: the idea is to use high-level utility functions as much as possible, while at the same time offering low-level primitives that those high-level ones are based on. If it can be made simpler, I'd really like to know. not that i have something better to offer. it's readable and looks fine. besides, i'm really sux at design things. ;-) signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote: sadly, this seems to get no wow!s. :-( If no one knows about it, wow!s will not happen. Make PRs. -Wyatt
Re: Make dub part of the standard dmd distribution
On Tue, 02 Jun 2015 22:30:47 +1200, Rikki Cattermole wrote: * non-windows, as dll support on windows is still not here, i believe. I've been able to do it a little bit already with my Web server. Haven't hit its limit so far. In other words as long as exceptions aren't used and its never unloaded, should be ok. Anyway, do you really think we would get that sort of support for a couple of releases? it seems that nobody on windows really needs that. ;-) Indeed lets. But can we also not go modifying the language? I'm just suggesting injecting of symbols that execute compiler side actions. Your suggesting language changes. Even if it is a new pragma. actually, no language changes at all. CTFE is already here, and it is used in semantic analysis stage. there are no changes to the language, besides adding some hooks and defining an API. it's invisible from the user's POV and doesn't require changing of specs. now CTFE is started explicitly, by using mixins and template evaluation. with my idea it will be started implicitly when some condition is met. sure, that conditions must be documented, but it will not change the language drastically. We really need to toy with these ideas properly and implement each. Then its just a matter of time to convince the higher ups that it should be merged. yes, i believe that this is the best approach. implement both and let the best solution win in a honest competition. ;-) i don't feel very creative right now, but implementing that idea is in my TODO list, and it's not at the bottom of list. so eventually (month? two? ten years?) i'll write a PoC. signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On 6/2/15 3:12 AM, Jacob Carlborg wrote: On 2015-06-01 20:49, Steven Schveighoffer wrote: I'm not defending DIP11, just that the fact that it could be done without requiring an extra dependencies file. One usually needs a file to set all compiler and linker flags and other kinds of configurations. Dub allows you to do this as well in cross-platform way. So when you already have all that, just put the dependencies there and be done with it. Actually, I don't. dmd *.d generally works fine. -Steve
Re: Make dub part of the standard dmd distribution
On 06/02/2015 02:53 AM, Jacob Carlborg wrote: On 2015-06-01 20:39, Nick Sabalausky wrote: Unlike rdmd, dub always tries to compile all sources regardless of whether they're actually imported. This has been a constant source of problems for me, including breakage of conditional importing under various circumstances. Isn't that the only way to compile it if the project is a library? without having to use workarounds like create a new module which imports all other modules. My libs always have a package.d or equivalent anyway, so I don't find it to be any sort of workaround. Even without that, rdmd-style still works if the lib is being used as a source lib rather than a pre-compiled static (or dynamic) lib. But yea, if you're doing a precompiled lib and you don't want any sort of root module (don't see it as all that big of a deal though), then yea I guess you'd have to collect the list of sources dub-style instead of rdmd-style.
Re: Make dub part of the standard dmd distribution
On 06/02/2015 01:03 AM, Nick Sabalausky wrote: On 06/01/2015 03:59 PM, Nick Sabalausky wrote: On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote: rdmd could collect dependencies while performing the actual compilation. Then, if the dependencies are stable (compare with the saved ones), all done in one run. Otherwise, discard the output and rebuild. Hmm, actually, that's a good idea. This can be implemented today. Would you want to work on it? I've gotten rusty on RDMD's internals, but I'll take a look, see what I can manage. Unfortunately, I've hit a bit of a problem: dmd -v doesn't emit an import module_name (path/to) line for imports that were provided on the command line. Aside from the minor issue of it complicating parts of the code, this makes it impossible (AFAICT) to distinguish between the deps have not changed vs a former dep is no longer imported and should no longer be compiled. Related to this, does anyone happen to recall why rdmd uses dmd -v to get dependencies instead of dmd -deps? IIRC, it used to use -deps back at one time.
Re: Make dub part of the standard dmd distribution
On 1/06/2015 5:57 p.m., Manu via Digitalmars-d wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d digitalmars-d@puremagic.com wrote: On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote: On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote: As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find. I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions. Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid? That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :) With dub you specify the dependencies in the dub config file, not in some obscure section of an INSTALL file as a command the users need to run. You can checkout a project using dub and with a single command have dub download and build all the dependencies (and their dependencies) and then build your project against them. I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically. dub is about making it easy for 99% of users. If you need your own build system then using dub just to download packages is overkill. Use git submodules or add something to do a download of your dependencies from github as part of your custom build system. Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing. I'm not trying to heckle. I just want someone to declare the word on where in the filesystem we put .d files, parallel to c includes. I think the single most important thing is for bindings against C libs that are installed by common -dev packages, it would be easiest if the bindings were fetched and installed the same way as the libs they refer to. Perhaps a new dub command (--locate)? $ dub build package --version=version --arch=arch --compiler=compiler --locate Base: ~/.dub/package-version/ Binary: ~/.dub/package-version/output.lib ImportPath: ~/.dub/package-version/source Pass Well you get the general gist. Fetches and builds if necessary for specific arch ext. Really it just minimizes output and only output location info.
Re: Make dub part of the standard dmd distribution
On 05/31/2015 11:48 PM, Manu via Digitalmars-d wrote: As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find. Ditto. Dub's great if you let it be your buildsystem, but as soon as you want to use it as a package-manager-only it becomes an uphill battle with dub fighting back every step of the way (I speak from experience). That problem is worse if your project is a library that you want fetchable through dub. That's why I've been working on some PRs to fix all that. The first step is here (building/expanding on a flag another contributer recently added): https://github.com/D-Programming-Language/dub/pull/572 (Although I think I may need to change --data=libs and separate system libs from dependency libs since linkers expect those to be passed in differently.) The next step will be to add a flag to have that data get output in a compiler-ready argument list (your choice of compiler, of course). That'll let you do something comparable to pkg-config --cflags --libs. Then finally, a shortcut flag so you can just do something kinda like: rdmd --build-only `dub describe --cmdargs=rdmd` src/main.d And all your dependencies will be properly referenced (-I..., linker flags, etc).
Re: Make dub part of the standard dmd distribution
On 06/01/2015 01:21 AM, Andrei Alexandrescu wrote: I installed a Python program the other day. I think it's rather obscure - it's called lcinvestor. I got from knowing nothing about how to install Python stuff to having the application running in minutes. That doesn't even remotely match my experience trying to use/install python-based software/libs: https://semitwist.com/articles/article/view/why-i-hate-python-or-any-dynamic-language-really
Re: Make dub part of the standard dmd distribution
On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote: Let's make this part of 2.068: https://issues.dlang.org/show_bug.cgi?id=14636 It's preapproved. Who would want to work on it? IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0.
Re: Make dub part of the standard dmd distribution
On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :) Not sure how serious/joking you are about that, but when has declaring a standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config.
Re: Make dub part of the standard dmd distribution
On 1 Jun 2015 08:45, Nick Sabalausky via Digitalmars-d digitalmars-d@puremagic.com wrote: On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :) Not sure how serious/joking you are about that, but when has declaring a standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config. Leave it to the distribution is the safe option in my experience. To have something along the lines of what Manu wants, I guess we need something like virtualenv to allow building in a clean/standard environment.
Re: Make dub part of the standard dmd distribution
On Monday, 1 June 2015 at 06:35:38 UTC, Nick Sabalausky wrote: On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote: Let's make this part of 2.068: https://issues.dlang.org/show_bug.cgi?id=14636 It's preapproved. Who would want to work on it? IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not certain exactly what work he had in mind for that though, versus post-1.0. DUB has worked equally well and almost unchanged for me in 2015.
Re: Make dub part of the standard dmd distribution
On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 Jun 2015 08:45, Nick Sabalausky via Digitalmars-d digitalmars-d@puremagic.com wrote: On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :) Not sure how serious/joking you are about that, but when has declaring a standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config. Leave it to the distribution is the safe option in my experience. To have something along the lines of what Manu wants, I guess we need something like virtualenv to allow building in a clean/standard environment. Yeah, I think I can see 2 parallel problems here. 1. There is a lib installed from a -dev package managed by the distribution... I just want the complementary .d files. (this is what I actually care about) 2. There is some open-source D code which isn't distributed as a binary, it's just a git repo and you fetch it and build it locally. (I find that I rarely need this, so I don't have much opinion on this case) For case 1, my preference would be a distro managed package alongside the lib itself, and installed into a standard location. If dub could pull the bindings I want and put them in some common location, fine. For case 2... I dunno. What if you offer a lib that falls into case 2; source is available, user can fetch and build against it locally, but it contains C code too? dub isn't exactly a build system which can express a complex build environment. I can't create a dub package for my engine, which might be of interest to D gamedevs.
Re: Make dub part of the standard dmd distribution
On 1 Jun 2015 07:57, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d digitalmars-d@puremagic.com wrote: On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote: On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote: As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find. I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions. Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid? That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :) With dub you specify the dependencies in the dub config file, not in some obscure section of an INSTALL file as a command the users need to run. You can checkout a project using dub and with a single command have dub download and build all the dependencies (and their dependencies) and then build your project against them. I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically. Just to be clear, libs are source libraries, right? dub is about making it easy for 99% of users. If you need your own build system then using dub just to download packages is overkill. Use git submodules or add something to do a download of your dependencies from github as part of your custom build system. Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing. This is a feature of your distribution, and not the language itself. I'm having talks with the Debian toolchain maintainer, we want to start shipping D programs and libraries with Debian/Ubuntu. Binary libraries are going to be the most interesting problem here because dmd and ldc will be shut out from using them. This is a semi call-out to the ldc devs, we should really align our ABIs together.
Re: Make dub part of the standard dmd distribution
On 1 Jun 2015 09:09, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 Jun 2015 08:45, Nick Sabalausky via Digitalmars-d digitalmars-d@puremagic.com wrote: On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :) Not sure how serious/joking you are about that, but when has declaring a standard whatever like that ever worked for anything ever? ;) Linux can't even agree with Linux on where things should go, apparently that's why C on linux has pkg-config. Leave it to the distribution is the safe option in my experience. To have something along the lines of what Manu wants, I guess we need something like virtualenv to allow building in a clean/standard environment. Yeah, I think I can see 2 parallel problems here. 1. There is a lib installed from a -dev package managed by the distribution... I just want the complementary .d files. (this is what I actually care about) 2. There is some open-source D code which isn't distributed as a binary, it's just a git repo and you fetch it and build it locally. (I find that I rarely need this, so I don't have much opinion on this case) For case 1, my preference would be a distro managed package alongside the lib itself, and installed into a standard location. If dub could pull the bindings I want and put them in some common location, fine. For case 2... I dunno. What if you offer a lib that falls into case 2; source is available, user can fetch and build against it locally, but it contains C code too? dub isn't exactly a build system which can express a complex build environment. I can't create a dub package for my engine, which might be of interest to D gamedevs. In one sense this can be solved at the distribution level. If dub was provided through your package manager, the package maintainers can ensure that dub was configured to understand where all system sources are located (or will be located). Can all of dubs default settings be dumped to json and be overriden?
Re: Make dub part of the standard dmd distribution
dub as a build tool sux. no need to discuss that, it simply sux. why do you think it sucks ? dub is a good package manager, but it's not a good build system. For small projects that just want to list dependencies and have it build, it's sufficient. Any larger project will have needs that won't be easily solved by using dub. Which is why I wrote my own. Atila
Re: Make dub part of the standard dmd distribution
Writing to something that requires root is just doing it wrong IMO, unless the user specifically opts in to it. /usr/anything is thus wrong, it should all be done locally. My preference is actually right in the current directory, do it on a per-project level. But if you must go higher, the user's home directory is also acceptable.
Re: Make dub part of the standard dmd distribution
On Monday, 1 June 2015 at 10:24:46 UTC, ketmar wrote: On Mon, 01 Jun 2015 05:05:20 +, Brad Anderson wrote: dub is about making it easy for 99% of users. If you need your own build system then using dub just to download packages is overkill. Use git submodules or add something to do a download of your dependencies from github as part of your custom build system. now you killed dub for normal GNU/Linux programmer. dub as a build tool sux. no need to discuss that, it simply sux. why do you think it sucks ? and now you killed the only sensible way to use it: as package manager. that's why, for example, i never considered adding anything to dub repository: that repository is completely useless for me. and other people who aren't use dub to build their projects will ignore dub repository too.
Re: Make dub part of the standard dmd distribution
On Monday, 1 June 2015 at 12:24:18 UTC, Atila Neves wrote: dub as a build tool sux. no need to discuss that, it simply sux. why do you think it sucks ? dub is a good package manager, but it's not a good build system. For small projects that just want to list dependencies and have it build, it's sufficient. Any larger project will have needs that won't be easily solved by using dub. Which is why I wrote my own. Atila Hi Atila, you are talking about cross language builds ? Other than that limitation, what do you miss ? I have a couple of rather sizable projects running with dub without problems and one could argue vibe.d is a pretty big project too which works fine. The D community already suffered enough from not uniting workforces behind at least SOME standards (hint at tango), a plethora of options for newcomers to choose from for basics like building is also daunting IMHO. Stephan
Re: Make dub part of the standard dmd distribution
On Monday, 1 June 2015 at 07:19:28 UTC, Iain Buclaw wrote: In one sense this can be solved at the distribution level. If dub was provided through your package manager, the package maintainers can ensure that dub was configured to understand where all system sources are located (or will be located). Can all of dubs default settings be dumped to json and be overriden? The way I do it right now is simply adding -I/usr/include/dlang -I/usr/include/dlang/compiler to dmd.conf / ldc.conf / patch GDC in a similar way. And surely if dub would ever attempt to write anything in /usr on its own I will simply throw it away from Arch packages.
Re: Make dub part of the standard dmd distribution
On Monday, 1 June 2015 at 07:42:02 UTC, Iain Buclaw wrote: Our druntimes need to be aligned up. The first thing you'd notice is that each may give undefined references to things the other compiler does not include in it's runtime. Unifying GDC/LDC ABI would simplify packager life a lot.
Re: Make dub part of the standard dmd distribution
On Mon, 01 Jun 2015 05:05:20 +, Brad Anderson wrote: dub is about making it easy for 99% of users. If you need your own build system then using dub just to download packages is overkill. Use git submodules or add something to do a download of your dependencies from github as part of your custom build system. now you killed dub for normal GNU/Linux programmer. dub as a build tool sux. no need to discuss that, it simply sux. and now you killed the only sensible way to use it: as package manager. that's why, for example, i never considered adding anything to dub repository: that repository is completely useless for me. and other people who aren't use dub to build their projects will ignore dub repository too. signature.asc Description: PGP signature
Re: Make dub part of the standard dmd distribution
On 1 June 2015 at 17:09, Iain Buclaw via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 Jun 2015 07:57, Manu via Digitalmars-d digitalmars-d@puremagic.com wrote: On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d digitalmars-d@puremagic.com wrote: On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote: On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote: As for dub, I'd use it if it worked like a package manager; dub get libcurl-d libqt-d zlib-d libsdl2-d etc I have no use for it as a build system, and therefore it's expression of dependencies is no use to me. I just want something that works the same way as '-dev' packages already work perfectly well in linux, that is, they fetch headers and libs, and put them in a standard location that all the tooling can find. I thought it does that. If dub doesn't allow me to type one command to download and install all I need about a package, we need to add that pronto. I consider it a dealbreaker. Andrei dub fetch does this already (though probably not quite what you are thinking of). You'd need to specify the paths manually because if it installed them to the global compiler paths we'd have dependency hell (what if 5 projects I have need 3 different versions of a library?). Also, you'd need root permissions. Yeah, but regardless, that's what I want. I don't have version hell with C libs distributed this way...? Is this a problem that people are specifically trying to avoid? That's not really how you use dub though. dub simply isn't a good fit for people who want it to be a system package manager. Its goals are different. If people want that they should work on getting libraries added to their preferred system's package registries. Right, so, someone decide a path, we'll write it on dlang.org, and then everyone will agree and fall in line :) With dub you specify the dependencies in the dub config file, not in some obscure section of an INSTALL file as a command the users need to run. You can checkout a project using dub and with a single command have dub download and build all the dependencies (and their dependencies) and then build your project against them. I get it, it sounds great... if your app suits the model. I have no D-only projects, all my programs combine many languages and ecosystems. There are also existing build systems that are well established that I prefer to use, integrate with IDE's, etc. I don't mind if people use dub, but I just want a way to fetch libs that the compilers will then find automatically. Just to be clear, libs are source libraries, right? Well, it should support libs that aren't JUST source. Binary libs needs some .d files to declare the API and all that, so at least some source would always present. A lib on its own us no use :) But any solution needs to support closed-source libraries too. Not everything can be conveniently distributed as full-source, and many libs are bindings against binary C libs which would probably be sourced externally. dub is about making it easy for 99% of users. If you need your own build system then using dub just to download packages is overkill. Use git submodules or add something to do a download of your dependencies from github as part of your custom build system. Point is, I don't have to do this with C. I just install the dev package, once, and I'm done. Package manager distributes updates automatically, everything it exactly how I want it. It's just not a wheel I have any interest in reinventing. This is a feature of your distribution, and not the language itself. I'm having talks with the Debian toolchain maintainer, we want to start shipping D programs and libraries with Debian/Ubuntu. Do want! :) Binary libraries are going to be the most interesting problem here because dmd and ldc will be shut out from using them. ? Why is? This is a semi call-out to the ldc devs, we should really align our ABIs together. Oh, are the LDC/GDC ABI's not consistent on linux? Surely many/most libs are just references to C '-dev' packages as dependencies?
