Re: Speed up `dub`.
On Sunday, 5 June 2016 at 21:20:20 UTC, Andrej Mitrovic wrote:
On Thursday, 2 June 2016 at 13:04:00 UTC, ciechowoj wrote:
and found that an assert from `std/path.d:3168` (`globMatch`)
contributes a major amount to the running time of dub.
```
assert(balancedParens(pattern, '[', ']', 0));
assert(balancedParens(pattern, '{', '}', 0));
```
Hmm.. that sounds like the dub binary that's distributed is
running its unittests at start. If that's true I don't think
they should be part of the released binary..
Actually, that assertions are invoked from precondition block of
`globMatch`. On the other hand `globMatch` is invoked from
`std.file.dirEntry`. So, I suppose it is fine if it goes about
unit tests.
Re: Speed up `dub`.
On Thursday, 2 June 2016 at 13:04:00 UTC, ciechowoj wrote:
and found that an assert from `std/path.d:3168` (`globMatch`)
contributes a major amount to the running time of dub.
```
assert(balancedParens(pattern, '[', ']', 0));
assert(balancedParens(pattern, '{', '}', 0));
```
Hmm.. that sounds like the dub binary that's distributed is
running its unittests at start. If that's true I don't think they
should be part of the released binary..
Re: Speed up `dub`.
On 04/06/16 10:04, Russel Winder via Digitalmars-d-learn wrote: Dub must check all dependencies before starting a build, the question is whether is should download and build them or just fail. Yeah, I meant check online. -- /Jacob Carlborg
Re: Speed up `dub`.
On Tue, 2016-05-24 at 09:00 +0200, Jacob Carlborg via Digitalmars-d- learn wrote: > […] > I think that by default Dub should not check dependencies. There > should > be an explicit command to install the dependencies. Dub must check all dependencies before starting a build, the question is whether is should download and build them or just fail. Gradle currently checks and downloads as required. This can be very annoying in that there is an assumption of "always connected" and "infinite data allowance". This can be very, very annoying when not connected or connected with a very limited connection. On the other hand, the build never fails due to ailed dependencies unless the dependency doesn't actually exist in the search path. For JVM language builds most people put both Bintray and Maven Central in the path and more or less all sensible dependencies live in one of those place – for FOSS stuff of course, for proprietary you are on your own :-) Gradle does now though build C++ stuff so there is already a dependency system. I am likely to try creating a D plugin for Gradle and then copy the Gradle way of doing things for SCons. I wonder of Dub should focus on using the same repositories for dependencies as Gradle does, not to avoif the Dub repository so much as to hijack native code build of C++ and ensure those people have D on offer. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: Speed up `dub`.
On 2016-06-02 15:07, ciechowoj wrote: Maybe it would be worth to write something about the issue here https://github.com/dlang/dub#installation . I'm curious how the pre-compiled versions are built. Yeah, I really hope they're built with optimizations enabled. -- /Jacob Carlborg
Re: Speed up `dub`.
I am not sure but on Arch linux I always recompile dub myself because by default it is build without release flags too. Dne 2.6.2016 v 15:07 ciechowoj via Digitalmars-d-learn napsal(a): Maybe it would be worth to write something about the issue here https://github.com/dlang/dub#installation . I'm curious how the pre-compiled versions are built.
Re: Speed up `dub`.
Maybe it would be worth to write something about the issue here https://github.com/dlang/dub#installation . I'm curious how the pre-compiled versions are built.
Re: Speed up `dub`.
On Thursday, 2 June 2016 at 12:20:50 UTC, Jacob Carlborg wrote:
On 2016-06-01 23:33, ciechowoj wrote:
Hahahaa. Who could possibly think that `build.sh` builds dub
in debug
mode? With -release -O -inline -m64 it runs 5 times faster :
P. It made
my day...
Haha really?
Yes, I should have done it at the beginning, but yesterday I
decided to see what is really happening and compiled dub with
`-profile` and found that an assert from `std/path.d:3168`
(`globMatch`) contributes a major amount to the running time of
dub.
```
assert(balancedParens(pattern, '[', ']', 0));
assert(balancedParens(pattern, '{', '}', 0));
```
Re: Speed up `dub`.
On 2016-06-01 23:33, ciechowoj wrote: Hahahaa. Who could possibly think that `build.sh` builds dub in debug mode? With -release -O -inline -m64 it runs 5 times faster : P. It made my day... Haha really? -- /Jacob Carlborg
Re: Speed up `dub`.
Hahahaa. Who could possibly think that `build.sh` builds dub in debug mode? With -release -O -inline -m64 it runs 5 times faster : P. It made my day...
Re: Speed up `dub`.
On 2016-03-07 10:18, ciechowoj wrote: I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time. I think that by default Dub should not check dependencies. There should be an explicit command to install the dependencies. -- /Jacob Carlborg
Re: Speed up `dub`.
On Thursday, 19 May 2016 at 17:50:44 UTC, ciechowoj wrote: dub build --nodeps I tried it, doesn't seem to do anything, maybe something is broken. Perhaps you didn't complete "dub build" without --nodeps beforehand? You have to do that once, and it's still as annoyingly inefficient as anything, especially since it checks dependencies online even if you have the selected version downloaded already. But after that first compile, I add --nodeps to the dub build command, and it's lightning fast to recompile. It doesn't fix the issues in DUB that slow down compiling programs, but it does speed up the compile-modify-check cycle.
Re: Speed up `dub`.
On Sunday, 15 May 2016 at 01:48:41 UTC, cy wrote: dub build --nodeps It's amazing. I tried it, doesn't seem to do anything, maybe something is broken.
Re: Speed up `dub`.
On Sunday, 15 May 2016 at 06:49:00 UTC, Russel Winder wrote: I am not sure about DStep build, but for the two projects I have (using GtkD) build times using Dub were sufficiently long that I switched to SCons and it is OK and allows for a reasonable development rhythm. That SCons looks very nice but don't have time (and dub isn't that bad) to research it more/can't migrate to it. If I have more time I'll certainly try it out.
Re: Speed up `dub`.
On Sat, 2016-05-14 at 16:36 +, ciechowoj via Digitalmars-d-learn wrote: > On Monday, 7 March 2016 at 21:56:11 UTC, Seb wrote: > > > > Use ld.gold - it will speed up your linking quite dramatically! > > > > https://code.dawg.eu/reducing-vibed-turnaround-time-part-1-faster-l > > inking.html > Thanks, it improves things a little. However I've just had idea > that it should be possible to implement 'speculative dependency > checking' for dub. What I mean by that is to start build process > and dependency checking in parallel and then if something changes > (with dependencies), apply the changes and restart the build > process, and if dependencies are OK continue. Anecdotal evidence from SCons, Gant, Gradle, Rant, and indeed Make, seems to indicate that full dependency resolution in the ADG of the build is the right thing to do for consistent and repeatable builds. Various debates have been had in the SCons and Gradle communities over the years about slow builds, and always, in the end, dependency resolution up front has always won out. Interestingly though "dependency resolution" actually has slightly different meanings in different contexts. In the SCons context it is about ensuring the ADG of the source → object → executable|library and the presence of any referenced libraries. In the Ant/Maven/Gant/Gradle context is is about downloading references libraries and ensuring the ADG of build tasks is right. In all cases building the ADG and constructing the resolution list is not the big problem except in very, very large builds. For example folk at Intel created Parts as a layer over SCons so as to parallelize and create partial caches for the build, a bit along the lines of Vesta but far better. I cannot be certain, but I am not sure there are any D codebases quite the same size as the Intel C++ ones. With SCons and Gradle there were various algorithmic and implementation speedups to ameliorate things and that has helped a lot. I wonder if there are opportunities to do the same for Dub. There are many functionality things that need to be added to Dub to get it into the Cargo and Gradle end of functionality mayhap performance can be improved along the way. I am not sure about DStep build, but for the two projects I have (using GtkD) build times using Dub were sufficiently long that I switched to SCons and it is OK and allows for a reasonable development rhythm. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: Speed up `dub`.
On Monday, 7 March 2016 at 09:18:37 UTC, ciechowoj wrote: I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time. dub build --nodeps It's amazing. dub has a few uh, issues with dependencies. It pulls all dependencies, even for disabled configurations, even for unselected optional dependencies, then just... leaves them pulled I guess, and doesn't link with them? https://github.com/dlang/dub/issues/844 I don't know exactly what's going wrong. But after you run `dub build` the first time, everything will be downloaded and installed now, so you can recompile by specifying --nodeps and it just jumps straight past all those buggy issues. ld.gold also will vastly speed up modify-compile-check cycles. dmd is fast at compiling, but linking it all up is really slow with vanilla ld.
Re: Speed up `dub`.
On Monday, 7 March 2016 at 21:56:11 UTC, Seb wrote: Use ld.gold - it will speed up your linking quite dramatically! https://code.dawg.eu/reducing-vibed-turnaround-time-part-1-faster-linking.html Thanks, it improves things a little. However I've just had idea that it should be possible to implement 'speculative dependency checking' for dub. What I mean by that is to start build process and dependency checking in parallel and then if something changes (with dependencies), apply the changes and restart the build process, and if dependencies are OK continue.
Re: Speed up `dub`.
On Tuesday, 8 March 2016 at 15:43:28 UTC, rcorre wrote: On Monday, 7 March 2016 at 22:47:13 UTC, Luis wrote: I try to grab dstep with dub fetch step (dub version 0.9.24, built on Aug 19 2015, on Ubuntu), and try to run dub build step. I can confirm that on my machine takes around 5 seconds to check if the dependencies are update. I had this same experience. Thought dub would be a convenient way to install dstep, but dub run took so long I ended up just creating an alias to run the binary directly. dub not should have something to install executables without the need of using dub run to execute it ? Like pip install does a true install of the package.
Re: Speed up `dub`.
On Monday, 7 March 2016 at 21:49:59 UTC, xcvn wrote: Yes but if he wants to build then `--describe` will not help that much ! dub uses that internally for finding the files for compiling, so thats the cause of the slowdown
Re: Speed up `dub`.
On Monday, 7 March 2016 at 18:58:55 UTC, ciechowoj wrote: dub --version DUB version 0.9.24+161-gb9ce700, built on Feb 23 2016 `dub.json` is `dub.json` of dstep `dub test --skip-registry=all` Do not helps. I try to grab dstep with dub fetch step (dub version 0.9.24, built on Aug 19 2015, on Ubuntu), and try to run dub build step. I can confirm that on my machine takes around 5 seconds to check if the dependencies are update.
Re: Speed up `dub`.
On Monday, 7 March 2016 at 09:18:37 UTC, ciechowoj wrote: I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time. Use ld.gold - it will speed up your linking quite dramatically! https://code.dawg.eu/reducing-vibed-turnaround-time-part-1-faster-linking.html
Re: Speed up `dub`.
On Monday, 7 March 2016 at 19:58:19 UTC, WebFreak001 wrote: On Monday, 7 March 2016 at 18:58:55 UTC, ciechowoj wrote: dub --version DUB version 0.9.24+161-gb9ce700, built on Feb 23 2016 `dub.json` is `dub.json` of dstep `dub test --skip-registry=all` Do not helps. Its because of dub going through all those JSON files. For example just querying the import paths would probably take like 7s in your case. Try "dub describe --import-paths" in your project. Its something you can't do anything against except reporting it to the dub issues Yes but if he wants to build then `--describe` will not help that much !
Re: Speed up `dub`.
On Monday, 7 March 2016 at 18:58:55 UTC, ciechowoj wrote: dub --version DUB version 0.9.24+161-gb9ce700, built on Feb 23 2016 `dub.json` is `dub.json` of dstep `dub test --skip-registry=all` Do not helps. Its because of dub going through all those JSON files. For example just querying the import paths would probably take like 7s in your case. Try "dub describe --import-paths" in your project. Its something you can't do anything against except reporting it to the dub issues
Re: Speed up `dub`.
I would say dub is broken and should be fixed. Even dub run is really slow and try to build everything. So please report a bug: https://github.com/D-Programming-Language/dub/issues Dne 7.3.2016 v 19:58 ciechowoj via Digitalmars-d-learn napsal(a): dub --version DUB version 0.9.24+161-gb9ce700, built on Feb 23 2016 `dub.json` is `dub.json` of dstep `dub test --skip-registry=all` Do not helps.
Re: Speed up `dub`.
dub --version DUB version 0.9.24+161-gb9ce700, built on Feb 23 2016 `dub.json` is `dub.json` of dstep `dub test --skip-registry=all` Do not helps.
Re: Speed up `dub`.
On Monday, 7 March 2016 at 09:18:37 UTC, ciechowoj wrote: I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time. A few hours ago dub.pm alias code.dlang.org was down or slow, so I used: $> dub --skip-registry=all build might be thats the reason?
Re: Speed up `dub`.
On Monday, 7 March 2016 at 09:18:37 UTC, ciechowoj wrote: I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time. On my case, I don't see taking too long. Could you check what version of dub are you using, and what OS & D compiler and version ? And better, if you can past here your dub.sdl/dub.json .
Re: Speed up `dub`.
On Monday, 7 March 2016 at 09:22:16 UTC, Daniel Kozak wrote: maybe: dub build --nodeps Dne 7.3.2016 v 10:18 ciechowoj via Digitalmars-d-learn napsal(a): I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time. I've tried that one already - it makes no difference. Currently when I run `dub test` it takes about 9s (project is already built). When running the executable directly takes only 0.2s :/.
Re: Speed up `dub`.
maybe: dub build --nodeps Dne 7.3.2016 v 10:18 ciechowoj via Digitalmars-d-learn napsal(a): I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time.
Speed up `dub`.
I'm using `dub` to build project. And every time I run `dub` it seems to check if dependencies are up to date, which takes some time. Is there a way to switch of that checking? Or any other way to speed up building process? It really slows down my modify-compile-check iteration time.
