Re: version(StdDoc)

2018-11-23 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, November 23, 2018 11:22:24 PM MST Neia Neutuladh via Digitalmars-
d-learn wrote:
> On Fri, 23 Nov 2018 21:43:01 -0700, Jonathan M Davis wrote:
> > A solution like that might work reasonably well, but you still
> > have the problem of what to do when a symbol is documented in multiple
> > version blocks, and having almost all the documentation in one version
> > block and a few pieces of it in other version blocks would risk getting
> > confusing and messy.
>
> Keeping symbol names and function arguments consistent between them is
> also an issue; it's not just the documentation. The normal solution is to
> put the version blocks inside the relevant symbols -- sometimes type
> aliases inside version blocks and consistent code outside, sometimes
> functions where the entire body is a set of version blocks.

When you're versioning the implementation, it's trivial to just version the
function internals. Where version(D_Ddoc) becomes critical is with stuff
like structs or enums where the members actually differ across systems. And
while it's generally better to try to avoid such situations, there are
definitely situations where there isn't much choice. Fortunately, in the
vast majority of situations, versioning across systems isn't required at
all, and in most of the situations where it is, its only the implementation
that needs to differ, but that's not true in all cases.

I do wish though that it were legal to use version blocks in more situations
than is currently the case. For instance,

enum Foo
{
a,
b,
version(linux) l,
else version(Windows) w,
}

is not legal, nor is

enum Foo
{
a,
b,
version(linux) c = 42,
else version(Windows) c = 54,
}

You're forced to version the entire enum. Fortunately, structs and classes,
do not have that restriction, so having to version an entire struct or class
at once is usually only required when the type needs to be declared on some
systems but not others (as is the case with WindowsTimeZone), and such
situations are rare.

- Jonathan M Davis





Re: version(StdDoc)

2018-11-23 Thread Neia Neutuladh via Digitalmars-d-learn
On Fri, 23 Nov 2018 21:43:01 -0700, Jonathan M Davis wrote:
> A solution like that might work reasonably well, but you still
> have the problem of what to do when a symbol is documented in multiple
> version blocks, and having almost all the documentation in one version
> block and a few pieces of it in other version blocks would risk getting
> confusing and messy.

Keeping symbol names and function arguments consistent between them is 
also an issue; it's not just the documentation. The normal solution is to 
put the version blocks inside the relevant symbols -- sometimes type 
aliases inside version blocks and consistent code outside, sometimes 
functions where the entire body is a set of version blocks.


Re: D is supposed to compile fast.

2018-11-23 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, November 23, 2018 11:13:24 AM MST H. S. Teoh via Digitalmars-d-
learn wrote:
> All in all, though, the fact that we're complaining about extra seconds
> in compilation times still does show just how fast D compilation can be.
> In the old days, compiling large C++ codebases usually means 30 mins to
> 2 hours, and a few extra seconds won't even be noticed.  I haven't
> checked C++ compile times recently, though -- things may have improved
> since I last seriously used C++.

When I was a student, I worked for a while at a company that had a large C++
code base that took over 3 hours to compile from scratch (incremental builds
were an absolute must). I ended up working somewhere else for a while and
then coming back again, and in the interim, they had begun redoing the
program in Java. The build process then took about 10 minutes, and folks
were complaining about it taking too long. After having been away for a
while, my perspective was that it was a _huge_ improvement, but since they'd
been working with it for a while, the 10 minutes was annoying. So, a lot of
it comes down to perspective.

D is often a _huge_ improvement when you first switch to it, but depending
on what your code does, over time, it can go from under a second to a few
seconds in compilation time, and for some folks that becomes maddening in
spite of the fact that the overall build times are a huge improvement over
what they would have had in another language - and usually, the features
that slow down the build the most are ones that don't even exist in other
languages (or if they do, are far less powerful). That being said, if we can
reasonably improve the compiler and standard library such that D code in
general builds faster with all of the CTFE and templates, we definitely
should.

- Jonathan M Davis





Re: version(StdDoc)

2018-11-23 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, November 23, 2018 7:55:04 PM MST H. S. Teoh via Digitalmars-d-
learn wrote:
> Adam does have a very good point about showing all alternatives to docs,
> though.  Arguably, that's what ddoc *should* do.  If the programmer
> wrote a ddoc comment in the code, it probably should be processed as
> part of doc generation, regardless of whether that code sits in some
> deeply-nested version blocks that ends up not being compiled.  Binding
> ddoc generation to the compile process seems not such a good idea in
> retrospect.

Honestly, I would argue that if you have multiple versions of the
documentation, then there's a serious problem. The documentation shouldn't
be platform-dependent even if the symbols are. Even if the documentation
needs some extra notes for a specific platform, it should all be in one
documentation block that anyone using the symbol can read. Providing
different documentation for different platforms just leads to folks not
understanding how the symbol differs across platforms, leading to code that
is even more platform-dependent when it really should be as platform
independent as possible.

The only situation I can think of at the moment where anything along the
lines of combining documentation across platforms makes sense would be if
there is a nested symbol that exists on only one platform (e.g. a member
function of a struct or a member of an enum). In that case, one platform
would have the main documentation, and then system-specific symbols would be
documented in those version blocks - but only those symbols. A solution like
that might work reasonably well, but you still have the problem of what to
do when a symbol is documented in multiple version blocks, and having almost
all the documentation in one version block and a few pieces of it in other
version blocks would risk getting confusing and messy. As such, I'm not sure
that the fact that ddoc forces you to have a separate set of declarations
just for the documentation is really a bad thing. It puts all of the
documentation in one place.

The bigger problem IMHO is how -D affects the build. Both it and -unittest
have the fundamental problem that because they create their own version
identifiers, they really shouldn't be part of the normal build, and yet the
way that they're set up to be used, it's as if they're expected to be part
of the normal build - with -D just causing the compiler to generate the
documentation in addition to the binary, and -unittest making the unit tests
run before main rather than replacing main.

At this point, I really wish that at minimum, -D were not set up to be part
of the normal build process so that version(D_Ddoc) would not affect it. The
same with -unittest. Ideally, it really would have replaced main rather than
putting the unit tests before it, with it providing main if one wasn't
there. That still doesn't solve all of the problems when using
version(unittest), but I doubt that much of anyone really wants to be
running unit tests as part of their application, and having such flags
clearly designed to create special builds rather than being designed such
that they could be used with the normal build would have fixed certain
classes of problems.

As for the issue of versioning the documentation, I don't really see a clean
one. Having the documentation build affected by version and static if and
the like causes some problems, but having it ignore them would likely cause
other problems. Regardless, I suspect that having the version identifiers
and static ifs be ignored almost requires a separate tool from the compiler
(such as Adam's documentation generator or ddox), because it's pretty clear
that ddoc was set up to generate the documentation for symbols as the
compiler compiles them, whereas a tool that ignored version identifiers and
static ifs would be processing the file quite differently.

- Jonathan M Davis





Re: version(StdDoc)

2018-11-23 Thread Jonathan M Davis via Digitalmars-d-learn
On Friday, November 23, 2018 2:47:51 PM MST Tony via Digitalmars-d-learn 
wrote:
> In std.compiler there is this code:
>
>  /// Which vendor produced this compiler.
>  version(StdDdoc)  Vendor vendor;
>  else version(DigitalMars) Vendor vendor = Vendor.digitalMars;
>  else version(GNU) Vendor vendor = Vendor.gnu;
>  else version(LDC) Vendor vendor = Vendor.llvm;
>  else version(D_NET)   Vendor vendor = Vendor.dotNET;
>  else version(SDC) Vendor vendor = Vendor.sdc;
>  else  Vendor vendor = Vendor.unknown;
>
> What is the situation in which the identifier StdDoc is set?

It's a Phobos-only replacement for the D_Ddoc version identifier and is
defined as part of the Phobos build. As the others have been complaining
about in this thread, ddoc is set up so that it takes whatever version is
currently being compiled. So, version(D_Ddoc) is the standard way to provide
documentation that is agnostic of the platform in the cases where a
documented symbol doesn't exist on all platforms, or its definition differs
such that it needs separate declarations - e.g. if the values of members in
an enum differ across systems, then you're forced to provide completely
separate enum declarations for the entire enum. If you have separate
declarations on different platforms, then in general there are only three
choices

1. Duplicate the documentation.

2. Set it up so that your documentation can only be built on one platform.

3. Use version(D_Ddoc) to provide a separate declaration just for the
documentation.

There are some cases where you can put the documentation outside the version
block and have it work - e.g.

/// my documentation
version(linux) int foo(string bar) { ... }
else version(Windows) int foo(string bar) { ... }
else ...

but that only works when you only need to document the top-level symbol
(such as with a function). It does not work with anything that needs to have
symbols inside it documented (such as with a struct, class, or enum).

Fortunately, most code does need to be versioned like this (especially if
you're trying to write platform-independent code like we try to do with
Phobos), and often, when you do need to version stuff, it's inside
implementations, but sometimes it does affect the top level. std.file has
this problem in a few places as does std.datetime with WindowsTimeZone
(since it only exists on Windows). The solution the language provides is to
use version(D_Ddoc) to version such documentation. And that's what Phobos
used to do.

The problem is that once you start using version(D_Ddoc), you _must_ have a
separate documentation build. You can't build your documentation as part of
your normal build, because you'd be getting the version(D_Ddoc) stubs
instead of the correct implementation for your platform (or instead of
nothing at all if that platform isn't supposed to have that particular
symbol). Phobos already had a separate documentation build because of how it
generates the documentation for the website, but many people's projects do
not. Most projects don't ever need to use version(D_Ddoc), because they
don't have symbols that are platform-dependent, and so before Phobos started
using it, they could compile their documentation as part of their normal
build just fine, because version(D_Ddoc) wasn't in their code anywhere. But
as soon as Phobos started using it, their code broke. If they had had a
separate documentation build, then they wouldn't have had any problems, but
as it was, their code was broken.

So, for better or worse, rather than saying that it was just bad practice to
do the documentation build as part of your normal build (and I would
strongly argue that building the documentation as part of the normal build
is bad practice given how doing so defines a new version identifier that can
affect the build), it was decided that Phobos would stop using
version(D_Ddoc) as the language intended and instead use its own custom,
version identifier for the documentation - which is why we now have
version(StdDoc) in Phobos and version(CoreDdoc) in druntime. With this
change, anyone building with the -D flag as part of their normal build
doesn't end up with version(D_Ddoc) screwing up their code because of
druntime or Phobos. Some other project they depend on could screw it up, and
if they ever use version(D_Ddoc) in their code, they'll have trouble again,
but Phobos and druntime aren't going to mess them up in that regard.

And while version(D_Ddoc) is the standard solution, because of this general
problem with folks using -D with their normal build, it's arguably best
practice at this point to create your own version identifier for your own
documentation for any library that you release that needs to version its
documentation. Otherwise, anyone using your library will have the same
problems that resulted in Phobos switching from version(D_Ddoc) to
version(StdDdoc) - which if anything shows that things 

How to iterate getSymbolsByUDA

2018-11-23 Thread Eko Wahyudin via Digitalmars-d-learn

Hi all,
anyone know how to iterate getSymbolsByUDA ?

I try this code:

enum Attr;
struct A
{
@Attr int a;
int b;
}

//---
//1. Error: need `this` for `a` of type `int`
foreach(sym; getSymbolsByUDA!(A, Attr)){
writeln(sym.stringof);  
}


for(sizediff_t i=0; i

Re: version(StdDoc)

2018-11-23 Thread Adam D. Ruppe via Digitalmars-d-learn

On Saturday, 24 November 2018 at 02:55:04 UTC, H. S. Teoh wrote:
But then that would be reinventing what Adam has already done, 
right? :-D


Precisely, I already do all that. And people are even actually 
using it!


Re: version(StdDoc)

2018-11-23 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Nov 24, 2018 at 02:09:22AM +, Neia Neutuladh via 
Digitalmars-d-learn wrote:
> On Fri, 23 Nov 2018 17:21:25 -0800, H. S. Teoh wrote:
> > Ddoc may have its stink points, but in this case, the stuff inside
> > version(Windows) blocks simply isn't compiled, so you can't expect
> > ddoc to do much about it.  You can't just arbitrarily ddoc
> > everything inside version blocks, because then you'll end up with
> > ddocs for stuff inside version(none) and/or conflicting docs for
> > alternative declarations in, say, OS-specific version blocks, or
> > user-defined static if's.
> 
> That means that, instead of one site showing my project's
> documentation, I need a separate site for every combination of
> platform and version flags.  And users who care about differences
> between platforms and version flags need to manually cross-reference
> docs for every symbol and overload.
> 
> It's a pretty terrible status quo.

True.  I've tried to tackle this before, but conceded defeat after I
realized that dmd just can't handle it because of the way ddoc is
implemented.

Adam does have a very good point about showing all alternatives to docs,
though.  Arguably, that's what ddoc *should* do.  If the programmer
wrote a ddoc comment in the code, it probably should be processed as
part of doc generation, regardless of whether that code sits in some
deeply-nested version blocks that ends up not being compiled.  Binding
ddoc generation to the compile process seems not such a good idea in
retrospect.

Now that I think about this more carefully, I feel tempted to say that
we should write a standalone ddoc generator, or at least a standalone
module in dmd, that process *all* source code regardless of version
blocks and static ifs and what-not. The output should be properly tagged
(e.g., the path of version identifiers / static if conditions that lead
to a particular version of the doc) so that the formatting backend can
render it sensibly, e.g., as a HTML drop-down list of versions, or
whatever.

But then that would be reinventing what Adam has already done, right?
:-D


T

-- 
If I were two-faced, would I be wearing this one? -- Abraham Lincoln


Re: version(StdDoc)

2018-11-23 Thread Neia Neutuladh via Digitalmars-d-learn
On Fri, 23 Nov 2018 17:21:25 -0800, H. S. Teoh wrote:
> Ddoc may have its stink points, but in this case, the stuff inside
> version(Windows) blocks simply isn't compiled, so you can't expect ddoc
> to do much about it.  You can't just arbitrarily ddoc everything inside
> version blocks, because then you'll end up with ddocs for stuff inside
> version(none) and/or conflicting docs for alternative declarations in,
> say, OS-specific version blocks, or user-defined static if's.

That means that, instead of one site showing my project's documentation, I 
need a separate site for every combination of platform and version flags. 
And users who care about differences between platforms and version flags 
need to manually cross-reference docs for every symbol and overload.

It's a pretty terrible status quo.


Re: version(StdDoc)

2018-11-23 Thread Adam D. Ruppe via Digitalmars-d-learn

On Saturday, 24 November 2018 at 01:21:25 UTC, H. S. Teoh wrote:
Ddoc may have its stink points, but in this case, the stuff 
inside version(Windows) blocks simply isn't compiled


That is why I call it "poorly designed" and a major reason why I 
dropped it entirely and created my own doc generator from scratch 
(well, using libdparse, so not exactly scratch, but zero use of 
dmd's code).


You can't just arbitrarily ddoc everything inside version 
blocks, because then you'll end up with ddocs for stuff inside 
version(none) and/or conflicting docs for alternative 
declarations in, say, OS-specific version blocks, or 
user-defined static if's.


That is easy to handle, you just treat it like overloads and 
display the version info along with the rest of it if both are 
documented. My doc generator does this and it is very useful - 
you can show what are under special version specifiers, call out 
OS differences, and more.


And if there are conflicting docs, the user ought to be able to 
see that!




Re: version(StdDoc)

2018-11-23 Thread H. S. Teoh via Digitalmars-d-learn
On Sat, Nov 24, 2018 at 12:51:36AM +, Adam D. Ruppe via Digitalmars-d-learn 
wrote:
> On Friday, 23 November 2018 at 23:13:04 UTC, H. S. Teoh wrote:
> > There are a few cases where this is needed, e.g., to generate docs
> > for Windows-specific modules, since the website script is run on
> > Posix and the Windows APIs would not be compiled at all, leading to
> > empty docs.
> 
> Note that that is only because ddoc is a badly designed piece of
> garbage.  Good doc generators can handle that just fine.

Ddoc may have its stink points, but in this case, the stuff inside
version(Windows) blocks simply isn't compiled, so you can't expect ddoc
to do much about it.  You can't just arbitrarily ddoc everything inside
version blocks, because then you'll end up with ddocs for stuff inside
version(none) and/or conflicting docs for alternative declarations in,
say, OS-specific version blocks, or user-defined static if's.

Having said that, though, it would be nice if there was a way to tell
the compiler "please generate docs for Windows, even though you're
currently running on Posix".  Resorting to hacks like that just to
generate Phobos docs simply sux.


T

-- 
Notwithstanding the eloquent discontent that you have just respectfully 
expressed at length against my verbal capabilities, I am afraid that I must 
unfortunately bring it to your attention that I am, in fact, NOT verbose.


Re: version(StdDoc)

2018-11-23 Thread Adam D. Ruppe via Digitalmars-d-learn

On Friday, 23 November 2018 at 23:13:04 UTC, H. S. Teoh wrote:
There are a few cases where this is needed, e.g., to generate 
docs for Windows-specific modules, since the website script is 
run on Posix and the Windows APIs would not be compiled at all, 
leading to empty docs.


Note that that is only because ddoc is a badly designed piece of 
garbage. Good doc generators can handle that just fine.


Re: Integrate vibe.d into Windows Service

2018-11-23 Thread Andre Pany via Digitalmars-d-learn

On Friday, 23 November 2018 at 21:35:57 UTC, Andre Pany wrote:

On Friday, 23 November 2018 at 21:27:26 UTC, Kagamin wrote:
You don't need to initialize runtime because it's initialized 
by standard D startup code, but you need to attach threads 
that are not created by D or static constructors won't be run 
and only C-level code would work there.


Thank you, I wasn't aware of this. For references, I found this 
post which explains thread_attachThis. I assume you meant this 
function.


Kind regards
Andre


Post explaining thread_attachThis 
https://forum.dlang.org/thread/ounui4$171a$1...@digitalmars.com


Re: version(StdDoc)

2018-11-23 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Nov 23, 2018 at 09:53:59PM +, Adam D. Ruppe via Digitalmars-d-learn 
wrote:
> On Friday, 23 November 2018 at 21:47:51 UTC, Tony wrote:
> > What is the situation in which the identifier StdDoc is set?
> 
> When the phobos website is being compiled, its own makefile sets that.
> 
> It is basically a hack for website display.

Yes, it's a hack to make the website display something different from
what ddoc would actually generate from the real code.

There are a few cases where this is needed, e.g., to generate docs for
Windows-specific modules, since the website script is run on Posix and
the Windows APIs would not be compiled at all, leading to empty docs.


T

-- 
It's amazing how careful choice of punctuation can leave you hanging:


Re: version(StdDoc)

2018-11-23 Thread Adam D. Ruppe via Digitalmars-d-learn

On Friday, 23 November 2018 at 21:47:51 UTC, Tony wrote:

What is the situation in which the identifier StdDoc is set?


When the phobos website is being compiled, its own makefile sets 
that.


It is basically a hack for website display.



version(StdDoc)

2018-11-23 Thread Tony via Digitalmars-d-learn

In std.compiler there is this code:

/// Which vendor produced this compiler.
version(StdDdoc)  Vendor vendor;
else version(DigitalMars) Vendor vendor = Vendor.digitalMars;
else version(GNU) Vendor vendor = Vendor.gnu;
else version(LDC) Vendor vendor = Vendor.llvm;
else version(D_NET)   Vendor vendor = Vendor.dotNET;
else version(SDC) Vendor vendor = Vendor.sdc;
else  Vendor vendor = Vendor.unknown;

What is the situation in which the identifier StdDoc is set?


Re: Making external types available to mixins

2018-11-23 Thread Kagamin via Digitalmars-d-learn

On Saturday, 17 November 2018 at 17:58:54 UTC, John Chapman wrote:
The following code doesn't compile because the generated type 
name needs to be available inside the mixin's scope, whereas 
it's actually in another module.


auto makeWith(string className, Args…)(auto ref Args args) {
  mixin("return makeWith!(I", className, "Factory)(args);"); // 
Fowarded to implementation of makeWith below

}

auto makeWith(T, Args…)(auto ref Args args) … // This is the 
implementation


The idea is that users could type (for example) 
makeWith!`Calendar`(…) instead of the longer 
makeWith!ICalendarFactory(…).


Well, just have all factories in one module and import it, then 
they will be visible.


import allfactories;
auto makeWith(string className, Args…)(auto ref Args args) {
  mixin("return makeWith!(I", className, "Factory)(args);"); // 
Fowarded to implementation of makeWith below

}

Or predeclare make functions in factory modules

interface ICalendarFactory
{
  ...
}

alias makeCalendar=makeWith!ICalendarFactory;


Re: Integrate vibe.d into Windows Service

2018-11-23 Thread Andre Pany via Digitalmars-d-learn

On Friday, 23 November 2018 at 21:27:26 UTC, Kagamin wrote:
You don't need to initialize runtime because it's initialized 
by standard D startup code, but you need to attach threads that 
are not created by D or static constructors won't be run and 
only C-level code would work there.


Thank you, I wasn't aware of this. For references, I found this 
post which explains thread_attachThis. I assume you meant this 
function.


Kind regards
Andre


Re: Integrate vibe.d into Windows Service

2018-11-23 Thread Kagamin via Digitalmars-d-learn
You don't need to initialize runtime because it's initialized by 
standard D startup code, but you need to attach threads that are 
not created by D or static constructors won't be run and only 
C-level code would work there.


Re: Integrate vibe.d into Windows Service

2018-11-23 Thread Andre Pany via Digitalmars-d-learn

On Wednesday, 21 November 2018 at 15:05:31 UTC, Kagamin wrote:
Start vibe in another thread and return from ServiceMain, see 
https://docs.microsoft.com/en-us/windows/desktop/Services/service-servicemain-function also ideally you should report running state only after vibe initialized, opened sockets and started listening, before that it's not really running.


I tried to wrap WaitForSingleObject into a Fiber by using command 
runTask. In theory it should work fine but the service (app) just 
crash without any information on an empty runTask function.


Also calling listenHTTP crashes the service. Here I am able to 
catch the Error "eventcore.core static constructor didn't run!?".


I created a github issue with complete source code here 
https://github.com/vibe-d/vibe-core/issues/105


Kind regards
Andre


Re: D is supposed to compile fast.

2018-11-23 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Nov 23, 2018 at 05:37:46PM +, Adam D. Ruppe via Digitalmars-d-learn 
wrote:
> On Friday, 23 November 2018 at 17:21:46 UTC, H. S. Teoh wrote:
> > Are you using template-heavy Phobos functions?
> 
> Merely importing a Phobos module is liable to cost you a quarter
> second or more of compile time.

Yes, I'm aware of that.  It used to be the case that the mere importing
of std.format would add 2-3 seconds to your compile time, but I think
the worst of it has been fixed.  It's still not cheap to use std.format,
but it's improved from what it used to be.


> I just got my gui lib (22,000 lines of raw source, dscanner -sloc
> reports 12,000) compiling in 0.2s on Linux, down from 1.0s, by
> removing ALL the phobos imports. It is 0.6s on Windows, probably
> because it imports so many of the Windows headers.

Yeah, I find that avoiding certain Phobos modules like std.format or
std.regex really does help a lot in improving compile times.


> I like to say C style and Java style code compiles really fast... but
> you wanna beware of imports of the standard library, since it brings
> in a LOT of code, some of which is really slow to run (parts of it are
> slower than others, like std.algorithm is actually pretty fast to
> compile+use, but std.regex is brutally slow)

Well yes, std.algorithm is practically all templates, and relatively
small ones (except for bears like cartesianProduct, no thanks to yours
truly :-/), so even though std.algorithm.* is pretty big in terms of
LOC, you only pay for what you use.  But std.regex has basically
template-driven internals, and IIRC has static tables that are generated
by templates and/or CTFE, so it can really slow the compiler down.

The other big offender is std.uni, which imports some pretty large
templated tables that adds a few seconds to compile times.

All in all, though, the fact that we're complaining about extra seconds
in compilation times still does show just how fast D compilation can be.
In the old days, compiling large C++ codebases usually means 30 mins to
2 hours, and a few extra seconds won't even be noticed.  I haven't
checked C++ compile times recently, though -- things may have improved
since I last seriously used C++.


T

-- 
IBM = I'll Buy Microsoft!


Re: D is supposed to compile fast.

2018-11-23 Thread Adam D. Ruppe via Digitalmars-d-learn

On Friday, 23 November 2018 at 17:21:46 UTC, H. S. Teoh wrote:

Are you using template-heavy Phobos functions?


Merely importing a Phobos module is liable to cost you a quarter 
second or more of compile time.


I just got my gui lib (22,000 lines of raw source, dscanner -sloc 
reports 12,000) compiling in 0.2s on Linux, down from 1.0s, by 
removing ALL the phobos imports. It is 0.6s on Windows, probably 
because it imports so many of the Windows headers.



I like to say C style and Java style code compiles really fast... 
but you wanna beware of imports of the standard library, since it 
brings in a LOT of code, some of which is really slow to run 
(parts of it are slower than others, like std.algorithm is 
actually pretty fast to compile+use, but std.regex is brutally 
slow)


Re: D is supposed to compile fast.

2018-11-23 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Nov 23, 2018 at 08:57:57AM +, Chris Katko via Digitalmars-d-learn 
wrote:
> Any time I see people mention the benefits of D, I see "compile times"
> "compile times" "compile times" over and over.

D is extremely fast at compilation ... of C-like code. :-D

Anything involving heavy use of templates and/or CTFE will quickly slow
things down, sometimes by a lot.


> I'm using very modest amounts of templates, for a fairly small sized
> program (very early work toward a game), and I'm hitting ~15 seconds
> compile time in LDC and ~7 seconds in DMD. And I'm not even compiling
> with optimizations!

Are you using template-heavy Phobos functions?  Some parts of Phobos are
known to be extremely slow, e.g., std.format (which is indirectly
imported by std.stdio), due to the sheer amount of templates it uses.


[...]
> I keep putting stuff into new files, but it feels like it's compiling
> everything from scratch / not getting faster the way C++ does.

Are you still compiling everything in one command, or separately
compiling?  There's not much point (as far as compile times are
concerned) in splitting up into new files if you're still compiling
everything each time.


> And I'm not even bringing up the 800MB of RAM required because I dared
> to import std.regex. (On a laptop with 2 GB of RAM. RIP. If I dare to
> have tabs open, the compile time goes into the minutes thanks to
> swapping.)

Yeah, std.regex is known to be pretty nasty in terms of compile times /
memory usage, because its internals uses a LOT of templates.  There have
been efforts to fix / improve this, but I haven't kept up with the
developments, so I'm not sure where things are at now.

One caveat about std.regex, though: you may actually want to use runtime
regexen instead of ctRegex, in spite of the supposed performance
improvements; I recall reading somewhere that ctRegex actually isn't as
fast as you might think, yet it comes at a HUGE compile-time cost (and I
do mean HUGE...  it can significantly increase compile times for only
marginal or sometimes even negative runtime performance -- because of
template bloat).  I've stopped using ctRegex altogether and just been
initializing regexen with `static this()` instead.  Or sometimes even
just pure runtime regexen, because the cost of initializing once is
insignificant compared to subsequent repeated usage.


T

-- 
Too many people have open minds but closed eyes.


Re: how to remove duplicate code in functional style

2018-11-23 Thread Alex via Digitalmars-d-learn

On Friday, 23 November 2018 at 14:33:40 UTC, berni wrote:
I've got the following code, which works, but obviously 
contains duplication. Is there a way to move that 
"dissection_available?...:..." to the place, where it should be?



return dissection_available
?solution.dup
 .transposed.map!(a=>a.map!(b=>"?#."[b]).array)
 
.zip(dissection.dup.transposed.map!(a=>a.map!(b=>"X#.?"[b]).array))

 .map!(a=>a[0].to!string~"  "~a[1].to!string)
 .join("\n")
 .to!string
:solution.dup
 .transposed.map!(a=>a.map!(b=>"?#."[b]).array)
 .join("\n")
 .to!string;


solution and dissection are of type const(int[][]).


Isn't the standard way of removing duplicate code in functional 
style to write more small grained functions?


like

return solution
. initialCommonOperation
. someAction(dissection, dissection_available)
. finalCommonOperation;

auto commonInitialCommonOperation(T)(T input)
{
return 
solution.dup.transposed.map!(a=>a.map!(b=>"?#."[b]).array);

}

auto someAction(T, A)(T tmp, A dissection, bool 
dissection_available)

{
return dissection_available ? 
tmp.zip(dissection.dup.transposed.map!(a=>a.map!

(b=>"X#.?"[b]).array))
.map!(a=>a[0].to!string~"  "~a[1].to!string) : tmp;
}

auto finalCommonOperation(T)(T tmp)
{
return tmp.join("\n").to!string;
}

Don't know, if this compiles... but you get my point ;)


how to remove duplicate code in functional style

2018-11-23 Thread berni via Digitalmars-d-learn
I've got the following code, which works, but obviously contains 
duplication. Is there a way to move that 
"dissection_available?...:..." to the place, where it should be?



return dissection_available
?solution.dup
 .transposed.map!(a=>a.map!(b=>"?#."[b]).array)
 
.zip(dissection.dup.transposed.map!(a=>a.map!(b=>"X#.?"[b]).array))

 .map!(a=>a[0].to!string~"  "~a[1].to!string)
 .join("\n")
 .to!string
:solution.dup
 .transposed.map!(a=>a.map!(b=>"?#."[b]).array)
 .join("\n")
 .to!string;


solution and dissection are of type const(int[][]).


Re: D is supposed to compile fast.

2018-11-23 Thread Stefan Koch via Digitalmars-d-learn

On Friday, 23 November 2018 at 08:57:57 UTC, Chris Katko wrote:
Any time I see people mention the benefits of D, I see "compile 
times" "compile times" "compile times" over and over.


[...]


If you can share the code privately I can use my custom profiling 
build of dmd to analyze the problem. just send me a mail 
(uplink{dot}coder{at}gmail{dot}com)


Re: memoize & __traits(compiles...)

2018-11-23 Thread Daniel Kozak via Digitalmars-d-learn
__traits(compiles...) does not call your function so it is not evaluate
twice only once, so there is no need to use memoize

On Fri, Nov 23, 2018 at 11:35 AM John Chapman via Digitalmars-d-learn <
digitalmars-d-learn@puremagic.com> wrote:

> I'm doing a fair amount of repeatedly checking if a function
> compiles with __traits(compiles...), executing the function if
> so, erroring out if not, like this:
>
>static if (__traits(compiles, generateFunc1())) {
>  return generateFunc1();
>} static if (__traits(compiles, generateFunc2())) {
>  return generateFunc2();
>} else static assert(false);
>
> But it seems inefficient to have to evaluate those functions
> twice, so I'd like to optimise this so if __traits(compiles...)
> succeeds, the result is cached and then used when the function is
> actually called. I wondered if using std.functional.memoize would
> help?
>


Re: memoize & __traits(compiles...)

2018-11-23 Thread John Chapman via Digitalmars-d-learn
On Friday, 23 November 2018 at 11:29:24 UTC, Nicholas Wilson 
wrote:
No, std.functional.memoize uses a hashtable to cache the 
runtime results of calls to expensive functions.


assuming that the example is not oversimplified and 
generateFunc1 and generateFunc2 are functions, the compiler 
doesn't do extra semantic analysis so the validity of the 
functions is effectively cached.


If they are templates (with parameters) then the compiler will 
automatically memoize them (it too keeps a hashtable of 
template instances).


Ah, that's good to know.


Re: memoize & __traits(compiles...)

2018-11-23 Thread Nicholas Wilson via Digitalmars-d-learn

On Friday, 23 November 2018 at 10:34:11 UTC, John Chapman wrote:
I'm doing a fair amount of repeatedly checking if a function 
compiles with __traits(compiles...), executing the function if 
so, erroring out if not, like this:


  static if (__traits(compiles, generateFunc1())) {
return generateFunc1();
  } static if (__traits(compiles, generateFunc2())) {
return generateFunc2();
  } else static assert(false);

But it seems inefficient to have to evaluate those functions 
twice, so I'd like to optimise this so if __traits(compiles...) 
succeeds, the result is cached and then used when the function 
is actually called. I wondered if using std.functional.memoize 
would help?


No, std.functional.memoize uses a hashtable to cache the runtime 
results of calls to expensive functions.


assuming that the example is not oversimplified and generateFunc1 
and generateFunc2 are functions, the compiler doesn't do extra 
semantic analysis so the validity of the functions is effectively 
cached.


If they are templates (with parameters) then the compiler will 
automatically memoize them (it too keeps a hashtable of template 
instances).


When in doubt, profile! 
https://blog.thecybershadow.net/2018/02/07/dmdprof/


Re: D is supposed to compile fast.

2018-11-23 Thread Daniel Kozak via Digitalmars-d-learn
On Fri, Nov 23, 2018 at 10:00 AM Chris Katko via Digitalmars-d-learn <
digitalmars-d-learn@puremagic.com> wrote:

> Any time I see people mention the benefits of D, I see "compile
> times" "compile times" "compile times" over and over.
>
> I'm using very modest amounts of templates, for a fairly small
> sized program (very early work toward a game), and I'm hitting
> ~15 seconds compile time in LDC and ~7 seconds in DMD. And I'm
> not even compiling with optimizations!
>
> ldc2 -w -ofextra  extra.d molto.d helper.d editor.d common.d
> map.d object_t.d animation.d ini.d  -L-L. $@-gc -d-debug=3
> -de -fdmd-trace-functions
>
> dmd -w -ofextra extra.d molto.d helper.d editor.d common.d map.d
> object_t.d animation.d ini.d -profile=gc  -profile  -g -debug
> -color -L-L.
>
> I keep putting stuff into new files, but it feels like it's
> compiling everything from scratch / not getting faster the way
> C++ does.
>
> And I'm not even bringing up the 800MB of RAM required because I
> dared to import std.regex. (On a laptop with 2 GB of RAM. RIP. If
> I dare to have tabs open, the compile time goes into the minutes
> thanks to swapping.)
>
>
AFAIK debug builds are slower? Can you share your files with us? So I can
try to find out what is the main problem


memoize & __traits(compiles...)

2018-11-23 Thread John Chapman via Digitalmars-d-learn
I'm doing a fair amount of repeatedly checking if a function 
compiles with __traits(compiles...), executing the function if 
so, erroring out if not, like this:


  static if (__traits(compiles, generateFunc1())) {
return generateFunc1();
  } static if (__traits(compiles, generateFunc2())) {
return generateFunc2();
  } else static assert(false);

But it seems inefficient to have to evaluate those functions 
twice, so I'd like to optimise this so if __traits(compiles...) 
succeeds, the result is cached and then used when the function is 
actually called. I wondered if using std.functional.memoize would 
help?


Re: Making external types available to mixins

2018-11-23 Thread John Chapman via Digitalmars-d-learn
On Thursday, 22 November 2018 at 16:27:08 UTC, Eduard Staniloiu 
wrote:

So I had a go at this and I have a working solution.
https://run.dlang.io/is/oaH6Ib

At first, I tried to do everything in the mixin, as you can see 
with the `failedAttempt` function. The idea was that this 
should have worked like `mixin(failedAttempt!"Calendar"(1, 2, 
3));`. As you can see, and the name suggests, I wasn't able to 
make it work with `args`.


The solution I have to your problem is to use a template, in 
this case the `theType` template that will expand to the fully 
qualified name. So you'd use it like

`makeWith!(theType!"Calendar")(args);`

Hope it helps!

Edi


Thanks!


Re: D is supposed to compile fast.

2018-11-23 Thread Nicholas Wilson via Digitalmars-d-learn

On Friday, 23 November 2018 at 08:57:57 UTC, Chris Katko wrote:
Any time I see people mention the benefits of D, I see "compile 
times" "compile times" "compile times" over and over.


I'm using very modest amounts of templates, for a fairly small 
sized program (very early work toward a game), and I'm hitting 
~15 seconds compile time in LDC and ~7 seconds in DMD. And I'm 
not even compiling with optimizations!


Templates are slow, slower than CTFE.

ldc2 -w -ofextra  extra.d molto.d helper.d editor.d common.d 
map.d object_t.d animation.d ini.d  -L-L. $@-gc -d-debug=3  
-de -fdmd-trace-functions


dmd -w -ofextra extra.d molto.d helper.d editor.d common.d 
map.d object_t.d animation.d ini.d -profile=gc  -profile  -g 
-debug -color -L-L.


I keep putting stuff into new files, but it feels like it's 
compiling everything from scratch / not getting faster the way 
C++ does.


If you pass all the files on the command line then they all get 
(re)compiled.
Have separate rules for each file, although if you are using 
template from each file in every file that may not help a whole 
lot.


And I'm not even bringing up the 800MB of RAM required because 
I dared to import std.regex. (On a laptop with 2 GB of RAM. 
RIP. If I dare to have tabs open, the compile time goes into 
the minutes thanks to swapping.)


Yep, on the upside the regexen are very fast at runtime. 
std.regex depending on how much of it you are using could also be 
a cause of your problem.you might want to take a look at 
https://blog.thecybershadow.net/2018/02/07/dmdprof/

https://github.com/CyberShadow/dmdprof to see whats taking so long

Is this code online? Its a bit difficult to give specific advice 
without it.


Re: Why does nobody seem to think that `null` is a serious problem in D?

2018-11-23 Thread Per Nordlöw via Digitalmars-d-learn

On Thursday, 22 November 2018 at 23:10:06 UTC, Per Nordlöw wrote:
With emphasis on _incremental_ additions to the compiler for 
covering more and more positives without introducing any 
_false_ negatives whatsoever. Without loosing compilation 
performance.


BTW, should such a compiler checking in D include pointers beside 
mandatory class checking?


D is supposed to compile fast.

2018-11-23 Thread Chris Katko via Digitalmars-d-learn
Any time I see people mention the benefits of D, I see "compile 
times" "compile times" "compile times" over and over.


I'm using very modest amounts of templates, for a fairly small 
sized program (very early work toward a game), and I'm hitting 
~15 seconds compile time in LDC and ~7 seconds in DMD. And I'm 
not even compiling with optimizations!


ldc2 -w -ofextra  extra.d molto.d helper.d editor.d common.d 
map.d object_t.d animation.d ini.d  -L-L. $@-gc -d-debug=3  
-de -fdmd-trace-functions


dmd -w -ofextra extra.d molto.d helper.d editor.d common.d map.d 
object_t.d animation.d ini.d -profile=gc  -profile  -g -debug 
-color -L-L.


I keep putting stuff into new files, but it feels like it's 
compiling everything from scratch / not getting faster the way 
C++ does.


And I'm not even bringing up the 800MB of RAM required because I 
dared to import std.regex. (On a laptop with 2 GB of RAM. RIP. If 
I dare to have tabs open, the compile time goes into the minutes 
thanks to swapping.)