Re: D Profile Viewer

[Andrew Trotman via Digitalmars-d-announce]

I've updated the d-profile-viewer.  There are three changes:
   Works on Windows (simple '\r' or '\n' bug fixed)
   Works with smaller numbers (division by integers -> divide by 
doubles)
   Uses HTML entities rather than UTF-8 (coz CodeWrite doesn't 
know UTF-8)


Its on dub: dub fetch d-profile-viewer
Its on bitbucket: 
https://bitbucket.org/andrewtrotman/d-profile-viewer


Thanks go to those who identified the bugs.

Andrew.

Re: dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 11:15:38 UTC, Joseph Rushton Wakeling 
wrote:
On Sunday, 15 May 2016 at 10:43:55 UTC, Joseph Rushton Wakeling 
wrote:
Probably the best way to handle this is to handle the 
take-the-address side of things by a @trusted wrapper that 
uses `return ref` to guarantee the pointer remains valid for 
the lifetime of the wrapper itself.


Note, I've been mulling over this myself for a while, so I'll 
probably put something together in a future dxorshift release 
(and probably try to get it in Phobos ASAP, as it will be very 
helpful in avoiding the worst cases of the existing RNG 
functionality).


Wrapper implemented here, together with documentation and tests:
https://github.com/WebDrake/dxorshift/pull/1

N.B. I'm sticking with the explicit wrapper, because I want to be 
really, really certain that what comes out is an input range 
whose underling RNG can _never_ be copied by value.

Re: [Semi OT] deadalnix inspires a C++ dependency-based coroutine scheduler

[deadalnix via Digitalmars-d-announce]

On Saturday, 14 May 2016 at 23:49:40 UTC, Ali Çehreli wrote:

Found on Reddit:


https://www.reddit.com/r/programming/comments/4jawhk/cosche_a_dependencybased_coroutine_scheduler_c/

The project:

  https://github.com/matovitch/cosche#cosche

The author says "I got the idea of building this by watching an 
amazing conference from Amaury Sechet on the Stupid D Compiler".


Ali


Fame and glory coming soon :)

Re: Found on Reddit: Cache sizes with CPUID in C++ and D

[Walter Bright via Digitalmars-d-announce]

On 5/15/2016 7:56 AM, Ali Çehreli wrote:

Then the author provides a function that does it correctly.


https://issues.dlang.org/show_bug.cgi?id=16028

Re: Found on Reddit: Cache sizes with CPUID in C++ and D

[Nordlöw via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 14:56:47 UTC, Ali Çehreli wrote:
Quote from the original article: "In D, there is a cpuid module 
that can retrieve cache information, but from what I've found 
it's incorrect."


Then the author provides a function that does it correctly.


But the article doesn't tell what's wrong with.

So what's wrong with:

import core.cpuid;
const(CacheInfo)[5] dc = dataCaches();

?

Re: Found on Reddit: Cache sizes with CPUID in C++ and D

[9il via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 14:56:47 UTC, Ali Çehreli wrote:


https://www.reddit.com/r/programming/comments/4jfoez/cache_sizes_with_cpuid_in_c_and_d/

Quote from the original article: "In D, there is a cpuid module 
that can retrieve cache information, but from what I've found 
it's incorrect."


Then the author provides a function that does it correctly.

Ali


Thanks!

Re: dxorshift: random number generators from the extended Xorshift family

[ag0aep6g via Digitalmars-d-announce]

On 05/15/2016 05:33 PM, Joseph Rushton Wakeling wrote:

Ah, interesting.  I think you may have discovered a bug in
`isForwardRange`, because that test _should_ have detected that, if
BaseRNG is a forward range, the RNG accessed via `alias this` is also
`save`able.


For a forward range, save must return the very same type. A forwarded 
`save` returns a different type.

Re: dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 15:33:24 UTC, Joseph Rushton Wakeling 
wrote:

I think you may have discovered a bug in `isForwardRange`


Less a bug than a subtlety, it seems.  Because of this line:
https://github.com/dlang/phobos/blob/778593d805a0c8bf39e318163e6d4004a7357904/std/range/primitives.d#L790

... the wrapper using `alias this` doesn't count as a forward 
range, because the result of `.save` is of the same type as the 
underlying RNG, rather than the wrapper itself.


The `.save` method still exists, though, although it's probably 
unlikely to be used in practice because `isForwardRange` 
evaluates to false.

Re: dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 15:15:30 UTC, Basile B. wrote:
I confirm that all of them are run. As in your original paste. 
All pass, 100% coverage. No problem. Anyway, NVM I should just 
take care of my own buisness...


Ah, interesting.  I think you may have discovered a bug in 
`isForwardRange`, because that test _should_ have detected that, 
if BaseRNG is a forward range, the RNG accessed via `alias this` 
is also `save`able.


static assert(!is(typeof(SafeRNG.save)));

... works, however.

Thank you very much for the suggestions here -- you showed up a 
nasty hole in the existing tests.

Re: dxorshift: random number generators from the extended Xorshift family

[Basile B. via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 14:49:16 UTC, Joseph Rushton Wakeling 
wrote:

On Sunday, 15 May 2016 at 14:25:44 UTC, Basile B. wrote:

The wrapper could be smaller with an alias this:

[... snip ...]

even if I'm not 100% sure if this is conform with previous 
version. At least the tests pass.


I'm surprised that one passes the test,

   static assert(!isForwardRange!SafeRNG);

... ?  Or did you only try the tests applied to the dxorshift 
generators?


I confirm that all of them are run. As in your original paste. 
All pass, 100% coverage. No problem. Anyway, NVM I should just 
take care of my own buisness...

Re: Battle-plan for CTFE

[Ola Fosheim Grøstad via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 15:09:17 UTC, Stefan Koch wrote:

You want it ?
Write it.


I don't «want» anything.

Re: Battle-plan for CTFE

[Stefan Koch via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 14:56:02 UTC, Ola Fosheim Grøstad wrote:


Well, this looks really bad.  But a solver would get you much 
more than an interpreter. E.g. proving that asserts always hold 
etc.


You want it ?
Write it.

Found on Reddit: Cache sizes with CPUID in C++ and D

[Ali Çehreli via Digitalmars-d-announce]

https://www.reddit.com/r/programming/comments/4jfoez/cache_sizes_with_cpuid_in_c_and_d/

Quote from the original article: "In D, there is a cpuid module that can 
retrieve cache information, but from what I've found it's incorrect."


Then the author provides a function that does it correctly.

Ali

Re: dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 14:25:44 UTC, Basile B. wrote:

The wrapper could be smaller with an alias this:

[... snip ...]

even if I'm not 100% sure if this is conform with previous 
version. At least the tests pass.


I'm surprised that one passes the test,

   static assert(!isForwardRange!SafeRNG);

... ?  Or did you only try the tests applied to the dxorshift 
generators?


More generally: what I'd be worried about with this version is 
that it readily re-opens the door to accidentally copying the 
underlying RNG by value, if (as with Phobos RNGs) the postblit is 
not disabled.


The wrapper I provided is more verbose, but it should guarantee 
statistical safety in that respect.


I also have a personal bugbear about `alias this` inasmuch as the 
current permissions constraints _force_ the exposure of 
implementation details (i.e. the `getGen` method has to be 
public), but that's more of a frustration than a serious worry in 
this case ;-)

Re: dxorshift: random number generators from the extended Xorshift family

[Basile B. via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 13:26:52 UTC, Joseph Rushton Wakeling 
wrote:
On Sunday, 15 May 2016 at 11:15:38 UTC, Joseph Rushton Wakeling 
wrote:
On Sunday, 15 May 2016 at 10:43:55 UTC, Joseph Rushton 
Wakeling wrote:
Probably the best way to handle this is to handle the 
take-the-address side of things by a @trusted wrapper that 
uses `return ref` to guarantee the pointer remains valid for 
the lifetime of the wrapper itself.


Note, I've been mulling over this myself for a while, so I'll 
probably put something together in a future dxorshift release 
(and probably try to get it in Phobos ASAP, as it will be very 
helpful in avoiding the worst cases of the existing RNG 
functionality).


Sneak preview to try out, before I tidy this up for actual 
inclusion in the library (needs docs, etc):


The wrapper could be smaller with an alias this:


public struct SafeUniformRNG (BaseRNG)
if (isUniformRNG!BaseRNG)
{
  public:
this (return ref BaseRNG generator) @trusted
{
this.gen = 
}

enum isUniformRandom = BaseRNG.isUniformRandom;

ref BaseRNG getGen()
{
return *gen;
}

alias getGen this;

  private:
BaseRNG* gen;

invariant()
{
assert(this.gen !is null);
}
}


even if I'm not 100% sure if this is conform with previous 
version. At least the tests pass.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/15/2016 04:00 PM, Daniel Murphy wrote:
> The problem is, if index refers to a single variable on the stack, then
> it's insufficient to refer to a variable inside an aggregate on the
> stack.  Then you need to start building constructs for member of struct
> in array of struct pointers and it gets fairly messy...  It's all
> solvable, I'm not sure the simplicity would survive.

Taking what I said further down below there would be one union Value
type (untagged b/c the compiler knows what it is).

Then an Array could be implementd as HeapValueRef[], a hash table as
HeapValueRef[ValueRef], a struct/class as HeapValueRef[] (with
HeapValueRef being a pointer or int to a heap allocated Value and
ValueRef being a pointer or int to either a stack value or a heap value).
A D Pointer/Ref would just be a ValueRef in the interpreter and the
aforementioned int (2B + for stack, 2B - for heap) encoding should still
work for that.
Depending on how much pointer arithmetic we support, Pointer must
contain a ValueRef to the outermost aggregate and needs to store the
type of that an additional byte offset. The type and offset could then
be used to compute the actual pointee. While that sounds a bit complex,
it's merely just https://en.wikipedia.org/wiki/Offsetof.

> Flow control is really not where the complexity lies IMO.  The weird
> ways in which different types of reference types can combine leads to
> either very complicated or very low level descriptions of memory.

Maybe you're right, but it'll be hard to figure out w/o an actual
implementation. And the AST still looks like a lot less to code and execute.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/15/2016 02:54 PM, Daniel Murphy wrote:
> 
> We really should have discussed this last week!

I talked about it w/ Stefan, and asked him to volunteer for an
implementation, that's why we have this thread ;).
In any case I'm convinced that the simple-first strategy has a much
higher chance to be implemented this year, whereas the bug report [¹]
for slow CTFE is already 5 years old.

[¹]: https://issues.dlang.org/show_bug.cgi?id=6498

Re: Battle-plan for CTFE

[Daniel Murphy via Digitalmars-d-announce]

On 15/05/2016 11:25 PM, Martin Nowak wrote:

On 05/15/2016 02:17 PM, Daniel Murphy wrote:


For simple types that's true.  For more complicated reference types...

Variable indexes are not enough, you also need heap memory, but slices
and pointers (and references) can refer to values either on the heap or
the stack, and you can have a slice of a member static array of a class
on the stack, etc.  Then there are closures...


So we do need a GC or RC for arrays, structs, classes (anything
heapish). Values for those could be allocated by a simple bump/region
allocator or a dedicated allocator that support individual freeing (via
RC or GC).

In any case struct Pointer { int index; /* 2B positive values for stack,
2B negative for heap*/ } wouldn't be much more complicated than a raw
pointer (and a bit simpler to garbage collect).



The problem is, if index refers to a single variable on the stack, then 
it's insufficient to refer to a variable inside an aggregate on the 
stack.  Then you need to start building constructs for member of struct 
in array of struct pointers and it gets fairly messy...  It's all 
solvable, I'm not sure the simplicity would survive.



Neither e2ir or s2ir are actually that complex.  A lot of the mess there
comes from the backend IR interface being rather difficult to work with.


Think of a simple switch statement where you even need to introduce
relocations (or keep a list of fixup addresses) b/c you don't know the
jump addresses in advance.


This is not exactly difficult to do.


In a visitor you simply test the cases and execute the first case body.

Not to mention that we can reuse existing solutions from the current
interpreter (e.g. for gotos see
https://github.com/dlang/dmd/blob/0757504342e48e272372b7ac52cda5a333b2a2bc/src/dinterpret.d#L1014
and
https://github.com/dlang/dmd/blob/0757504342e48e272372b7ac52cda5a333b2a2bc/src/dinterpret.d#L1094).



Flow control is really not where the complexity lies IMO.  The weird 
ways in which different types of reference types can combine leads to 
either very complicated or very low level descriptions of memory.

Re: Battle-plan for CTFE

[Stefan Koch via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 12:54:33 UTC, Daniel Murphy wrote:


We really should have discussed this last week!


I agree.
Maybe we can have a skype conference or something in the next 
days.


About the whole to BC or not to BC discussion.
As Daniel already outlined, the main purpose it not speed, but 
having a simple  lowered representation to interpret.
Many AST interpreters switched to a byte-code because it's much 
easier to maintain.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/15/2016 02:13 PM, Ola Fosheim Grøstad wrote:
> 
> Well, you can, but it won't bring improvements to the language down the
> line.

Maybe you don't know the actual problem of the current interpreter?
I leaks memory like hell b/c it allocates new AST nodes for almost every
expression evaluation.
Even an interpreter that is as slow as ruby will fly during compile time
in comparision to the current one.

Let me illustrate the point.

---
import std.algorithm, std.array, std.random, std.range;

enum count = 2 ^^ 10;
enum sorted = {
auto gen = Random(123);
return generate!(() => uniform(byte.min, byte.max,
gen)).take(count).array.sort().release;
}();
pragma(msg, sorted.length);
---
count = 2 ** 10
nums = Enumerator.new do |yielder|
  prng = Random.new(123)
  loop do
yielder.yield prng.rand(-128 .. 127)
  end
end.take(count).sort
print nums.length
---

   N   | CTFE | Ruby
   | Time  | Mem  | Time  | Mem
---|---|--|---|--
 2^^10 | 0.16s | 82M  | 0.11s | 9.3M
 2^^11 | 0.22s | 110M | 0.12s | 9.3M
 2^^12 | 0.4s  | 190M | 0.12s | 9.4M
 2^^13 | 0.7s  | 450M | 0.12s | 9.5M
 2^^14 | 1.5s  | 1.4G | 0.12s | 9.7M
 2^^15 | 3.7s  | 4.8G | 0.13s | 10.0M
 2^^16 | 5:30m | 15G  | 0.13s | 10.8M

D's CTFE grows O(N^2) b/c it leaks for almost every operation.
We don't currently need a superfast interpreter, even the simplest
possible interpreter will allow so much more that we're more likely
limited by the lack of I/O before we need a faster interpreter.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/15/2016 02:17 PM, Daniel Murphy wrote:
> 
> For simple types that's true.  For more complicated reference types...
> 
> Variable indexes are not enough, you also need heap memory, but slices
> and pointers (and references) can refer to values either on the heap or
> the stack, and you can have a slice of a member static array of a class
> on the stack, etc.  Then there are closures...

So we do need a GC or RC for arrays, structs, classes (anything
heapish). Values for those could be allocated by a simple bump/region
allocator or a dedicated allocator that support individual freeing (via
RC or GC).

In any case struct Pointer { int index; /* 2B positive values for stack,
2B negative for heap*/ } wouldn't be much more complicated than a raw
pointer (and a bit simpler to garbage collect).

> Neither e2ir or s2ir are actually that complex.  A lot of the mess there
> comes from the backend IR interface being rather difficult to work with.

Think of a simple switch statement where you even need to introduce
relocations (or keep a list of fixup addresses) b/c you don't know the
jump addresses in advance.
In a visitor you simply test the cases and execute the first case body.

Not to mention that we can reuse existing solutions from the current
interpreter (e.g. for gotos see
https://github.com/dlang/dmd/blob/0757504342e48e272372b7ac52cda5a333b2a2bc/src/dinterpret.d#L1014
and
https://github.com/dlang/dmd/blob/0757504342e48e272372b7ac52cda5a333b2a2bc/src/dinterpret.d#L1094).

Re: dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 11:15:38 UTC, Joseph Rushton Wakeling 
wrote:
On Sunday, 15 May 2016 at 10:43:55 UTC, Joseph Rushton Wakeling 
wrote:
Probably the best way to handle this is to handle the 
take-the-address side of things by a @trusted wrapper that 
uses `return ref` to guarantee the pointer remains valid for 
the lifetime of the wrapper itself.


Note, I've been mulling over this myself for a while, so I'll 
probably put something together in a future dxorshift release 
(and probably try to get it in Phobos ASAP, as it will be very 
helpful in avoiding the worst cases of the existing RNG 
functionality).


Sneak preview to try out, before I tidy this up for actual 
inclusion in the library (needs docs, etc):


/

module dxorshift.wrapper;

import std.random : isUniformRNG;

public struct SafeUniformRNG (BaseRNG)
if (isUniformRNG!BaseRNG)
{
  public:
this (return ref BaseRNG generator) @trusted
{
this.gen = 
}

enum isUniformRandom = BaseRNG.isUniformRandom;

bool empty() @property
{
return this.gen.empty;
}

auto front() @property
{
return this.gen.front;
}

auto popFront()
{
this.gen.popFront();
}

auto seed(Seed...)(Seed s)
{
this.gen.seed(s);
}

  private:
BaseRNG* gen;

invariant()
{
assert(this.gen !is null);
}
}

public auto uniformRNG(BaseRNG)(return ref BaseRNG generator)
{
return SafeUniformRNG!BaseRNG(generator);
}

// test `uniformRNG`'s behavior with phobos RNGs
@safe unittest
{
import std.array : array;
import std.random : isUniformRNG, isSeedable, PseudoRngTypes;
import std.range.primitives : isInputRange, isForwardRange;
import std.range : take;

foreach (RNG; PseudoRngTypes)
{
alias SafeRNG = SafeUniformRNG!RNG;

static assert(isUniformRNG!SafeRNG);
static assert(isSeedable!SafeRNG == isSeedable!RNG);

static assert(isInputRange!SafeRNG);
static assert(!isForwardRange!SafeRNG);

// assuming RNG is seedable, we validate
// expected differences between phobos
// RNGs' normal behaviour and how they
// behave when wrapped by `uniformRNG`
static if (isSeedable!RNG)
{
RNG gen;
gen.seed(123456);

// if we pass any normal phobos RNG
// directly into a range chain, it
// will (sadly) be copied by value
auto take1 = gen.take(10).array;
auto take2 = gen.take(10).array;
assert(take1 == take2);

gen.seed(123456);

// if however we wrap it with `uniformRNG`
// it will be passed by reference
auto safeGen = uniformRNG(gen);
auto take3 = safeGen.take(10).array;
auto take4 = safeGen.take(10).array;
assert(take3 == take1); // because we start from the 
same seed

assert(take3 != take4);

// validate we can however re-seed the
// safely wrapped generator and get
// the same results once again
safeGen.seed(123456);
auto take5 = safeGen.take(10).array;
auto take6 = safeGen.take(10).array;
assert(take5 == take3);
assert(take6 == take4);
}
}
}

// validate it works with dxorshift RNGs
// and allows them to work in @safe code
@safe nothrow pure unittest
{
import std.array : array;
import std.meta : AliasSeq;
import std.random : isUniformRNG, isSeedable;
import std.range.primitives : isInputRange, isForwardRange;
import std.range : take;

import dxorshift : SplitMix64, Xoroshiro128plus, 
Xorshift1024star;


foreach (RNG; AliasSeq!(SplitMix64, Xoroshiro128plus, 
Xorshift1024star))

{
alias SafeRNG = SafeUniformRNG!RNG;

static assert(isUniformRNG!SafeRNG);
static assert(isSeedable!SafeRNG);
static assert(isInputRange!SafeRNG);
static assert(!isForwardRange!SafeRNG);

// dxorshift generators must be constructed
// with a seed
auto gen = RNG(123456);

// we can't copy dxorshift RNGs by value,
// and it's not safe to just take the
// pointer address, so let's just jump
// to wrapping them in `uniformRNG`
auto safeGen = uniformRNG(gen);
auto take1 = safeGen.take(10).array;
auto take2 = safeGen.take(10).array;
assert(take1 != take2);

// re-seeding should give us the same
// results once over
gen.seed(123456);
auto take3 = safeGen.take(10).array;
auto take4 = safeGen.take(10).array;
assert(take3 == take1);
assert(take4 == take2);

// re-seeding via the safe wrapper
// should produce the same results
safeGen.seed(123456);
auto take5 = safeGen.take(10).array;
   

Re: Battle-plan for CTFE

[Daniel Murphy via Digitalmars-d-announce]

On 15/05/2016 9:57 PM, Martin Nowak wrote:

On 05/15/2016 01:58 PM, Daniel Murphy wrote:

The biggest advantage of bytecode is not the interpreter speed, it's
that by lowering you can substitute VarExps etc with actual references
to memory without modifying the AST.

By working with something lower level than the AST, you should end up
with something much less complex and with fewer special cases.


Which is a bad assessment, you can stick variable indexes into
VarDeclaration (we already do that) and thereby access them in O(1).
Converting control flow and references into byte code is far from
trivial, we're talking about another s2ir and e2ir here.

-Martin



We really should have discussed this last week!

Re: Battle-plan for CTFE

[Daniel Murphy via Digitalmars-d-announce]

On 15/05/2016 9:57 PM, Martin Nowak wrote:

On 05/15/2016 01:58 PM, Daniel Murphy wrote:

The biggest advantage of bytecode is not the interpreter speed, it's
that by lowering you can substitute VarExps etc with actual references
to memory without modifying the AST.

By working with something lower level than the AST, you should end up
with something much less complex and with fewer special cases.


Which is a bad assessment, you can stick variable indexes into
VarDeclaration (we already do that) and thereby access them in O(1).
Converting control flow and references into byte code is far from
trivial, we're talking about another s2ir and e2ir here.

-Martin



For simple types that's true.  For more complicated reference types...

Variable indexes are not enough, you also need heap memory, but slices 
and pointers (and references) can refer to values either on the heap or 
the stack, and you can have a slice of a member static array of a class 
on the stack, etc.  Then there are closures...


Neither e2ir or s2ir are actually that complex.  A lot of the mess there 
comes from the backend IR interface being rather difficult to work with. 
 We can already save a big chunk of complexity by not having to 
translate the frontend types.  E.g.  implementing the logic in the 
interpreter to correctly unwind through destructors is unlikely to be 
simpler than lowering to an IR.

Re: Battle-plan for CTFE

[Ola Fosheim Grøstad via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 12:00:43 UTC, Martin Nowak wrote:

On 05/15/2016 01:55 PM, Ola Fosheim Grøstad wrote:
If you are going to have fast evaluation of loops/recursion 
then you need to use a solver. And well, doing worse than 
O(log N) at compile time is a very bad idea.


Why not start with the most difficult case first? Then the 
simple cases will resolve themselves for free, most likely.


Why not do something that takes about a month and is much more 
likely to succeed?


Well, you can, but it won't bring improvements to the language 
down the line. If typical CTFE can be rewritten as simple tail 
recursion then you probably can find existing libraries that will 
do it for you.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/15/2016 02:02 PM, Stefan Koch wrote:
> 
> Correct. A ByteCode Interpreter will add even more implementation
> overhead, and the benefit is only realizable if the ByteCode  is a
> standard format that can be read other backends such as a jit.

This indeed would be an interesting proposal, interpretable IR that is
portable between the backends. But I guess we won't find such IR or at
least need to lower it into RealIR for dmd/gcc/llvm.
Sounds like an interesting candidate for the next GSoC.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/15/2016 01:55 PM, Ola Fosheim Grøstad wrote:
> If you are going to have fast evaluation of loops/recursion then you
> need to use a solver. And well, doing worse than O(log N) at compile
> time is a very bad idea.
> 
> Why not start with the most difficult case first? Then the simple cases
> will resolve themselves for free, most likely.

Why not do something that takes about a month and is much more likely to
succeed?
If someone has more time and needs an even faster interpreter she can
write a new one, or add optimizations or JIT to the simple interpreter.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/15/2016 01:58 PM, Daniel Murphy wrote:
> The biggest advantage of bytecode is not the interpreter speed, it's
> that by lowering you can substitute VarExps etc with actual references
> to memory without modifying the AST.
> 
> By working with something lower level than the AST, you should end up
> with something much less complex and with fewer special cases.

Which is a bad assessment, you can stick variable indexes into
VarDeclaration (we already do that) and thereby access them in O(1).
Converting control flow and references into byte code is far from
trivial, we're talking about another s2ir and e2ir here.

-Martin

Re: Battle-plan for CTFE

[Stefan Koch via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 10:29:21 UTC, Martin Nowak wrote:

On 05/10/2016 08:45 AM, Jacob Carlborg wrote:


I was listening to a discussion Don and Daniel had about the 
current implementation of CTFE. They talked about using a byte 
code interpreter. Even implementing a really crappy byte code 
interpreter would be a huge improvement.


No need for a byte-code interpreter, it mostly just adds 
overhead and complexity over an AST interpreter. If you want to 
go really fast you need some sort of JIT anyhow, but a proper 
interpreter will be orders of mangnitude faster than the 
current implementation.


I might refer you to
http://dconf.org/2013/talks/chevalier_boisvert.pdf
page 59 ff.


Correct. A ByteCode Interpreter will add even more implementation 
overhead, and the benefit is only realizable if the ByteCode  is 
a standard format that can be read other backends such as a jit.

Re: Battle-plan for CTFE

[Daniel Murphy via Digitalmars-d-announce]

On 15/05/2016 8:29 PM, Martin Nowak wrote:


No need for a byte-code interpreter, it mostly just adds overhead and
complexity over an AST interpreter. If you want to go really fast you
need some sort of JIT anyhow, but a proper interpreter will be orders of
mangnitude faster than the current implementation.



The biggest advantage of bytecode is not the interpreter speed, it's 
that by lowering you can substitute VarExps etc with actual references 
to memory without modifying the AST.


By working with something lower level than the AST, you should end up 
with something much less complex and with fewer special cases.


The current goal is not a full JIT, just something that manages memory 
in a less insane way.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/09/2016 06:57 PM, Stefan Koch wrote:
> I was shocked to discover that the PowExpression actually depends on
> phobos! (depending on the exact codePath it may or may not compile...)
> which let to me prematurely stating that it worked at ctfe
> [http://forum.dlang.org/thread/ukcoibejffinknrbz...@forum.dlang.org]

There is a really old bug report for that [Issue 3749 – cannot evaluate
yl2x (log) and exp functions at compile
time](https://issues.dlang.org/show_bug.cgi?id=3749). The lack of exp is
really limiting for many nice table precomputation use-cases in
scientific contexts.

-Martin

Re: Battle-plan for CTFE

[Ola Fosheim Grøstad via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 10:29:21 UTC, Martin Nowak wrote:

On 05/10/2016 08:45 AM, Jacob Carlborg wrote:
overhead and complexity over an AST interpreter. If you want to 
go really fast you need some sort of JIT anyhow, but a proper 
interpreter will be orders of mangnitude faster than the 
current implementation.


If you are going to have fast evaluation of loops/recursion then 
you need to use a solver. And well, doing worse than O(log N) at 
compile time is a very bad idea.


Why not start with the most difficult case first? Then the simple 
cases will resolve themselves for free, most likely.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/13/2016 06:32 PM, Stefan Koch wrote:
> I would like to work on a solution that does scale. The Problem is 
> not making a byteCode-interpreter. That part is relatively easy. 
> Currently I am trying to get a detailed understanding of dmd and 
> it's data-structures. (mainly it's AST.)
> 
> Generating the byte-code seems to be non-trivial.
> 
> I wonder in how far the glue layer can be of help...

Seems like I've to repeat this once more, b/c everyone including me
didn't got it in the first place. We don't need a bytecode
interpreter, it mostly adds overhead and a complicated second layer
between walking the AST and interpreting it (think of transforming a
for loop with goto into linear bytecode, almost as complicated as in
the glue layer).

What we basically need is a stack of values, a stack of frames (for
function calls and variables in scopes), and an AST visitor that does
the interpretation.
It would be most helpful for the success of this to follow common CS
examples like [¹], [²], or [³].
With realistic expectations we might have a working implementation in
a month or so. With more requirements like bytecode, using dmd's
backend, or JIT we end up with a long newsgroup discussion ;).

Tricky things for a CTFE interpreter include:

- enumerating VarDeclarations (they already have a ctfeAdrOnStack
field) in each scope, and referring to outer variables from nested scopes

At best just use a continuous stack and just set the stack pointer to
the last frame pointer when leaving a scope.

- getting function calls right

Push arguments, on return shift top of stack under arguments and pop
them (caller cleanup). If possible detect and support tail recursion.

- converting AST values to and from Interpreter Values.

Literals and constant VarExp from the AST need to be converted to an
interpreter Value before being pushed on the stack. The result of
interpretation (top of stack) needs to be converted back to an AST
literal.
Using separate data types (instead of creating AST values in the
interpreter) will be a major performance improvement over using AST
values (e.g. 16 vs. ~100 bytes). It also creates a clear boundary
between Interpreter and AST values. Currently quite some complexity is
thrown at cleaning interpreter generated AST values, and
distinguishing interpreter owned from normal AST values (which allows
some optimizations) [⁴].

We don't need a tagged union b/c the AST already contains the type
information, but a tag could be helpful for debugging [⁵].

Values can be deleted when popped from stack (no ref counting needed I
think).

- Implementing more complex data structures (arrays, strings, hash
tables, aggregates)

Use Value[], Value[Value], and a dedicated String
(char[]/wchar[]/dchar[]). For structs/classes field indexes are known
=> use fix-sized Value[]. Value.class_ needs to hold a reference to
the actual class instance for vtable calls.

Last time I was working on this (also on a bytecode interpreter) the
entry point was fairly clear [⁶] (thanks to Don).

-Martin

[¹]: [The Interpreter In An Undergraduate Compilers Course
](http://arxiv.org/pdf/1412.0426.pdf)
[²]: [L8: Interpreters &
Visitors](http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-005-elements-of-software-construction-fall-2011/lecture-notes/MIT6_005F11_lec08.pdf)
[³]: [PA 2:
Interpreter](https://sites.google.com/a/bodik.org/cs164/projects/pa2)
[⁴]: https://github.com/dlang/dmd/search?q=ownedByCtfe
[⁵]:
https://github.com/MartinNowak/dmd/blob/28ffb0ab4fa6950f60c085f33f8a2ce23df7c0cd/src/interpret.c#L73
[⁶]:
https://github.com/MartinNowak/dmd/blob/28ffb0ab4fa6950f60c085f33f8a2ce23df7c0cd/src/interpret.c#L693

Re: dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 10:43:55 UTC, Joseph Rushton Wakeling 
wrote:
Probably the best way to handle this is to handle the 
take-the-address side of things by a @trusted wrapper that uses 
`return ref` to guarantee the pointer remains valid for the 
lifetime of the wrapper itself.


Note, I've been mulling over this myself for a while, so I'll 
probably put something together in a future dxorshift release 
(and probably try to get it in Phobos ASAP, as it will be very 
helpful in avoiding the worst cases of the existing RNG 
functionality).

Re: It's alive! D building D building D, all on Android

[Joakim via Digitalmars-d-announce]

On Wednesday, 11 May 2016 at 19:07:10 UTC, Joakim wrote:

On Thursday, 5 May 2016 at 14:07:07 UTC, Vadim Lopatin wrote:

On Thursday, 5 May 2016 at 08:17:07 UTC, Joakim wrote:

[...]


Great work!


I've slapped up some beta builds, have at it:

https://github.com/joakim-noah/android/releases/tag/ddmd

Check out this beautiful version string, a natively built ddmd 
on my Android tablet:


LDC - the LLVM D compiler (1.0.0-beta1):
  based on DMD v2.070.2 and LLVM 3.8.0
  built with LDC - the LLVM D compiler (3197d6)
  Default target: armv7-none-linux-android
  Host CPU: cortex-a15
  http://dlang.org - http://wiki.dlang.org/LDC

  Registered Targets:
arm - ARM
armeb   - ARM (big endian)
thumb   - Thumb
thumbeb - Thumb (big endian)


I've put up three more builds, including ldc master, which uses 
the latest 2.071 frontend.  Once I get JNI and the sample app 
working, I'll make a proper announcement.

Re: dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Sunday, 15 May 2016 at 10:35:11 UTC, Basile B. wrote:
The "@disable this" is really a concern, because pointers have 
to be used (for example if the seed comes from a program option 
and that the gen is a global var then global var must be a 
pointer to the stuff).


I see that you are yourself affected by the issue because in 
the unittest you must take the gen address to use it in take   .


The main consequence is that they are unsable in @safe code !


The @safe side of things is obviously a concern, but having @safe 
code is not very helpful if you don't also have _statistical_ 
safety.  See what happens with phobos RNGs if you try,


import std.stdio : writeln;
import std.random : Random, unpredictableSeed
import std.range : take;

auto gen = Random(unpredictableSeed);

gen.take(10).writeln;
gen.take(10).writeln;

... ;-)

Probably the best way to handle this is to handle the 
take-the-address side of things by a @trusted wrapper that uses 
`return ref` to guarantee the pointer remains valid for the 
lifetime of the wrapper itself.

Re: Battle-plan for CTFE

[Martin Nowak via Digitalmars-d-announce]

On 05/10/2016 08:45 AM, Jacob Carlborg wrote:
> 
> I was listening to a discussion Don and Daniel had about the current
> implementation of CTFE. They talked about using a byte code interpreter.
> Even implementing a really crappy byte code interpreter would be a huge
> improvement.

No need for a byte-code interpreter, it mostly just adds overhead and
complexity over an AST interpreter. If you want to go really fast you
need some sort of JIT anyhow, but a proper interpreter will be orders of
mangnitude faster than the current implementation.

I might refer you to
http://dconf.org/2013/talks/chevalier_boisvert.pdf
page 59 ff.

dxorshift: random number generators from the extended Xorshift family

[Joseph Rushton Wakeling via Digitalmars-d-announce]

http://code.dlang.org/packages/dxorshift
https://github.com/WebDrake/dxorshift

Following my earlier list posting 
, I'm pleased to announce an initial release of a dub package providing some of the RNGs from the extended family of xorshift-inspired generators.  These should complement the existing xorshift implementations already provided to Phobos' std.random by Masahiro Nakagawa: they are fast, high-quality generators representing some of the state of the art in pseudo-RNG design.


The generators implemented have been ported from public-domain 
reference implementations available here: 
http://xoroshiro.di.unimi.it/  Following the original authors' 
example, these D ports have also been dedicated to the public 
domain using the Creative Commons CC0 license.


At this stage, only a direct port has been provided, with no 
attempts at generics (e.g. on the unsigned integer type used, or 
the magic constants used in the RNG update methods).  The 
provided generators are:


  * SplitMix64: a fast generator with only 64 bits
of state; this is probably inadequate for any
serious statistical work, but is provided as a
convenient means of generating seeds for other
more heavy-duty algorithms

  * xoroshiro128+: a very fast and high quality
generator with 128 bits of state; this ought
to be a great generator for anyone not doing
massively parallel simulations

  * xorshift1024*: a very fast and high quality
generator with 1024 bits of state; this is
slower than xoroshiro128+, but can be used
with much larger-scale parallel simulations

The xoroshiro128+ and xorshift1024* generators come with `jump()` 
methods, the equivalent of 2 ^^ 64 and 2 ^^ 512 calls to 
`popFront()` respectively, which can be used to generate the 
starting points for (again respectively) 2 ^^ 64 or 2 ^^ 512 
independent sequences of variates.


The generators are all implemented as structs, but in order to 
prevent some known problems with unintended copy-by-value of 
RNGs, the postblit has been disabled.  For similar reasons, these 
generators are implemented as input ranges, not forward ranges, 
so that library functionality cannot copy generator state under 
the hood.


`dup` properties are however provided for all generators, to 
allow the programmer to deliberately copy RNG state.


Testing, feedback and general usage are all welcome.  I am 
planning on submitting these to Phobos (although sorting out the 
generic side of things might be a good idea first).