Static array size?
Docs says that: "The total size of a static array cannot exceed 16Mb." But when I am creation array of: int [1000_000] x; // 1000_000 is equal ~ 0,95MB app crush on start. Should it's reserve this memory with guaranty? I mean that after app start it should take +0.95MB of RAM in task manager.
Re: Static array size?
V Thu, 09 Feb 2017 11:22:28 + Suliman via Digitalmars-d-learn napsáno: > Docs says that: > "The total size of a static array cannot exceed 16Mb." > But when I am creation array of: > int [1000_000] x; // 1000_000 is equal ~ 0,95MB > app crush on start. > > Should it's reserve this memory with guaranty? I mean that after > app start it should take +0.95MB of RAM in task manager. First of all it is 4bytes * 1000_000 = 4000_000 bytes = 4kbyte, and it should not crash, are you sure this is the reason why app crash?
Re: Static array size?
On Thursday, 9 February 2017 at 11:22:28 UTC, Suliman wrote: Docs says that: "The total size of a static array cannot exceed 16Mb." But when I am creation array of: int [1000_000] x; // 1000_000 is equal ~ 0,95MB app crush on start. Should it's reserve this memory with guaranty? I mean that after app start it should take +0.95MB of RAM in task manager. multiply by 4
Re: Static array size?
On Thursday, 9 February 2017 at 12:16:26 UTC, jkpl wrote: On Thursday, 9 February 2017 at 11:22:28 UTC, Suliman wrote: Docs says that: "The total size of a static array cannot exceed 16Mb." But when I am creation array of: int [1000_000] x; // 1000_000 is equal ~ 0,95MB app crush on start. Should it's reserve this memory with guaranty? I mean that after app start it should take +0.95MB of RAM in task manager. multiply by 4 Also static array is not on the heap so take care to which column of the task manager you monitor.
Re: Static array size?
On Thursday, 9 February 2017 at 12:13:36 UTC, Daniel Kozák wrote: V Thu, 09 Feb 2017 11:22:28 + Suliman via Digitalmars-d-learn napsáno: Docs says that: "The total size of a static array cannot exceed 16Mb." But when I am creation array of: int [1000_000] x; // 1000_000 is equal ~ 0,95MB app crush on start. Should it's reserve this memory with guaranty? I mean that after app start it should take +0.95MB of RAM in task manager. First of all it is 4bytes * 1000_000 = 4000_000 bytes = 4kbyte, and it should not crash, are you sure this is the reason why app crash? s/kbyte/megabyte
Re: Static array size?
On Thursday, 9 February 2017 at 11:22:28 UTC, Suliman wrote: Docs says that: "The total size of a static array cannot exceed 16Mb." But when I am creation array of: int [1000_000] x; // 1000_000 is equal ~ 0,95MB app crush on start. Should it's reserve this memory with guaranty? I mean that after app start it should take +0.95MB of RAM in task manager. First of all, as others said it is 4 bytes * 100, what gives about 4MB. Do you allocate this array on the stack? (local variables are created on the stack) If yes, maybe you've exceeded the size of the stack. Declare the array out of the scope of any function and check again. The size of the stack can be set in the linker (http://forum.dlang.org/post/mailman.766.1291253609.21107.digitalmar...@puremagic.com).
Factory using an alias template parameter to set a member of the new tool ?
I'm looking for a better way to do this, if possible:
```
class Tool
{
string name;
}
T namedTool(alias Variable, T)()
{
T result = new T;
result.name = Variable.stringof;
return result;
}
void main()
{
Tool grep;
grep = namedTool!(grep,Tool);
assert(grep.name == "grep");
}
```
Ideally this would work like this:
```
Tool grep = namedTool!Tool;
assert(grep.name == "grep");
```
Possible ?
Re: Factory using an alias template parameter to set a member of the new tool ?
On Thursday, 9 February 2017 at 13:30:07 UTC, jkpl wrote:
I'm looking for a better way to do this, if possible:
```
class Tool
{
string name;
}
T namedTool(alias Variable, T)()
{
T result = new T;
result.name = Variable.stringof;
return result;
}
void main()
{
Tool grep;
grep = namedTool!(grep,Tool);
assert(grep.name == "grep");
}
```
Ideally this would work like this:
```
Tool grep = namedTool!Tool;
assert(grep.name == "grep");
```
Possible ?
Sorry, but this does not make much sense to me ...
You expect "namedTool!Tool" to know that the name of the Tool
must be set to "grep".
The expression evaluation proceeds from right to left, so at the
time of analyzing "namedTool!Tool", the compiler knows nothing
about "grep".
On the other hand, if you supplied the "grep" variable, the
compiler could infer its type, like this:
```
...
auto namedTool(alias Variable)()
{
alias T = typeof(Variable);
T result = new T;
result.name = Variable.stringof;
return result;
}
void main()
{
Tool grep;
grep = namedTool!(grep);
assert(grep.name == "grep");
}
```
Re: Factory using an alias template parameter to set a member of the new tool ?
On Thursday, 9 February 2017 at 14:39:41 UTC, angel wrote:
On Thursday, 9 February 2017 at 13:30:07 UTC, jkpl wrote:
I'm looking for a better way to do this, if possible:
```
class Tool
{
string name;
}
T namedTool(alias Variable, T)()
{
T result = new T;
result.name = Variable.stringof;
return result;
}
void main()
{
Tool grep;
grep = namedTool!(grep,Tool);
assert(grep.name == "grep");
}
```
Ideally this would work like this:
```
Tool grep = namedTool!Tool;
assert(grep.name == "grep");
```
Possible ?
Sorry, but this does not make much sense to me ...
You expect "namedTool!Tool" to know that the name of the Tool
must be set to "grep".
The expression evaluation proceeds from right to left, so at
the time of analyzing "namedTool!Tool", the compiler knows
nothing about "grep".
On the other hand, if you supplied the "grep" variable, the
compiler could infer its type, like this:
```
...
auto namedTool(alias Variable)()
{
alias T = typeof(Variable);
T result = new T;
result.name = Variable.stringof;
return result;
}
void main()
{
Tool grep;
grep = namedTool!(grep);
assert(grep.name == "grep");
}
```
Or actually, maybe this will suite your case better:
```
template namedTool(T, alias Variable)
{
enum namedTool = T.stringof ~ " " ~ Variable ~ " = new " ~
T.stringof ~ ";" ~
Variable ~ ".name = \"" ~ Variable ~ "\";";
}
void main()
{
mixin(namedTool!(Tool, "grep"));
assert(grep.name == "grep");
}
```
Re: Factory using an alias template parameter to set a member of the new tool ?
On Thursday, 9 February 2017 at 15:00:21 UTC, angel wrote:
On Thursday, 9 February 2017 at 14:39:41 UTC, angel wrote:
On Thursday, 9 February 2017 at 13:30:07 UTC, jkpl wrote:
I'm looking for a better way to do this, if possible:
Or actually, maybe this will suite your case better:
```
template namedTool(T, alias Variable)
{
enum namedTool = T.stringof ~ " " ~ Variable ~ " = new " ~
T.stringof ~ ";" ~
Variable ~ ".name = \"" ~ Variable ~ "\";";
}
void main()
{
mixin(namedTool!(Tool, "grep"));
assert(grep.name == "grep");
}
```
Thanks for trying. I know that it doesn't make much sense but it
would have been a cool sugar for the tool class.
Mixins are not good for me because of completion in the IDE.
Strange behaviour of rdmd vs. dmd concerning main function
I've got two source files in two directories:
Common/common.d
module common;
import std.stdio;
int main(string[] args)
{
Foo foo = cast(Foo)Object.factory("special.Bar");
foo.do_something();
return 0;
}
abstract class Foo {
abstract void do_something();
}
Special/special.d
module special;
import std.stdio;
import common;
class Bar : Foo {
override void do_something()
{
writeln("works");
}
}
Now I try to run it with rdmd and dmd and get quite different
results:
$> rdmd -ICommon Special/special.d
works
$> dmd -ICommon Special/special.d
/usr/lib/gcc/x86_64-linux-gnu/6/../../../x86_64-linux-gnu/Scrt1.o: In function
`_start':
(.text+0x20): undefined reference to `main'
special.o:(.data.rel.ro+0x18): undefined reference to
`_D6common12__ModuleInfoZ'
special.o:(.data._D7special3Bar7__ClassZ+0x50): undefined
reference to `_D6common3Foo7__ClassZ'
collect2: error: ld returned 1 exit status
Error: linker exited with status 1
I encountered this, when I was curious if I can move the main
function to a diffent module, because it will be the same in
several modules anyway. When testing with rdmd, everything
worked. Later, I found out, that with dmd it doesn't. (And with
gdc and ldc2 it doesn't work too.)
For me, this looks like a bug in either rdmd or dmd. But maybe
there is something about it that I do not understand.
Re: Strange behaviour of rdmd vs. dmd concerning main function
On Thu, Feb 09, 2017 at 03:39:18PM +, berni via Digitalmars-d-learn wrote: > Now I try to run it with rdmd and dmd and get quite different results: dmd only compiles in the files you actually pass to it. rdmd will try to find the required files automatically. Since you didn't pass the file with the function to dmd, it knows it exists, but leaves it out of the final link (it assumes it might come from a library or something). That's why you see the error. rdmd exists because it handles it more automatically.
Re: Strange behaviour of rdmd vs. dmd concerning main function
dmd only compiles in the files you actually pass to it. rdmd will try to find the required files automatically. Since you didn't pass the file with the function to dmd, it knows it exists, but leaves it out of the final link (it assumes it might come from a library or something). That's why you see the error. Ah ok, I understand. So calling with "dmd Special/special.d Common/common.d" works. But when I compile common.d to common.o (with dmd -c common.d) and remove common.d after that is there still a possibility to link? The obvious doesn't work: $> dmd Special/special.d Common/common.o Special/special.d(4): Error: module common is in file 'common.d' which cannot be read import path[0] = /usr/include/dmd/phobos import path[1] = /usr/include/dmd/druntime/import
Re: Cross-compile with LDC
On Wednesday, 8 February 2017 at 17:57:49 UTC, kinke wrote: On Wednesday, 8 February 2017 at 17:21:03 UTC, Oleg B wrote: If I understand correctly with vanilla LDC I can't cross-compiling from host linux-x86_64, but with your patch I can. Right? Right. Joakim Noah has worked on LDC for Android and as far as I know provides some prebuilt compilers, a native one and a cross-compiler (Linux x86_64 host) including that patch. There's also a pre-built Windows -> ARM cross-compiler; see http://forum.dlang.org/thread/xzzzfvahuwvgsluli...@forum.dlang.org and more ARM-related threads in our subforum. That's not really a Windows build of ldc, he just used the new linux support in Windows 10. There is a build of ldc 1.1.0 for native linux/ARM, if you want to compile D code on the ARM device itself: https://github.com/ldc-developers/ldc/releases/tag/v1.1.0 If not, it's pretty straight forward to build a cross-compiler for ARM, take a look at my patches and instructions for Android: https://github.com/joakim-noah/android/releases If I use your patch must I use in my programs only double or I can stay real in existing code? Don't worry, the reals can stay. The patch will make reals the same as double, ie they're all 64-bit, so it is irrelevant which one you use. If you're writing cross-platform code and care about that difference on other platforms, you may need to check for it. Take a look at std.math for examples of this, where it checks the length of the mantissa sometimes for reals.
fix for sample mandelbrot code from documentation -- latest gdc has no lockingBinaryWriter
Hi,
I was trying to run the example code from
https://dlang.org/phobos/std_stdio.html#.File.lockingBinaryWriter
which is very short, as follows.
===
import std.algorithm, std.range, std.stdio;
void main()
{
enum size = 500;
writef("P5\n%d %d %d\n", size, size, ubyte.max);
iota(-1, 3, 2.0/size).map!(y =>
iota(-1.5, 0.5, 2.0/size).map!(x =>
cast(ubyte)(1+
recurrence!((a, n) => x + y*1i + a[n-1]^^2)(0+0i)
.take(ubyte.max)
.countUntil!(z => z.re^^2 + z.im^^2 > 4))
)
)
.copy(stdout.lockingBinaryWriter);
}
===
Unfortunately, my compiler fails with
$ gdc -Wall -o m1 m1.d m2.d:14:17: error: no property
'lockingBinaryWriter' for type 'File'
.copy(stdout.lockingBinaryWriter);
It looks like the latest in the repository isn't recent enough,
since the source code online to stdio.d on github has
lockingBinaryWriter in it, but my copy installed via ubuntu apt
does not.
I tried
$ gdc --version
and got
gdc (Ubuntu 6.2.0-5ubuntu12) 6.2.0 20161005
but that doesn't say what version the frontend compiler or
standard library is. After searching on the web, I found a
pragma that I added to the file
pragma(msg, __VERSION__);
which gave the version "2068L" when I compiled it. According to
https://gdcproject.org/downloads
the latest gdc(1) uses the DMDFE version 2.068.2 which isn't new
enough I guess, since the DMD version appears to be 2.073.0 which
includes lockingBinaryWriter.
This fixes the problem:
===
import std.range, std.array, std.stdio;
void main()
{
auto buf = appender!(ubyte[])();
enum size = 500;
writef("P5\n%d %d %d\n", size, size, ubyte.max);
iota(-1, 3, 2.0/size).map!(y =>
iota(-1.5, 0.5, 2.0/size).map!(x =>
cast(ubyte)(1+
recurrence!((a, n) => x + y*1i + a[n-1]^^2)(0+0i)
.take(ubyte.max)
.countUntil!(z => z.re^^2 + z.im^^2 > 4))
)
)
.copy(buf);
stdout.rawWrite(buf.data);
}
===
And running it
$ gdc -Wall -o m2 m2.d && ./m1 > aaa.pnm && display aaa.pnm
produces the expected result.
I'm posting this in case anyone else encounters this problem. I
don't know D at all, and perhaps this will save someone else from
some head scratching and going down output iterator rabbit holes.
Best regards,
Oasiq.
Can this implementation of Damm algorithm be optimized?
Hi,
I was trying to port C code from the article in Wikiversity [1]
to D, but I'm not sure this implementation is the most efficient
way to do it in D, so suggestions to optimize it are welcome:
import std.stdio;
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') + (tmpdigit
- '0') * 10];
return tmpdigit;
}
enum {
EXIT_SUCCESS = 0,
EXIT_FAILURE = 1,
}
int main(string[] args) {
scope(failure) {
writeln("Invalid arguments. You must pass a number.");
return EXIT_FAILURE;
}
assert(args.length == 2);
char digit = checkDigit(args[1]);
if(digit == '0') writefln("%s is a valid number.", args[1]);
else {
writefln("%s is not a valid number (but it would be,
appending digit %s).",
args[1], digit);
}
return EXIT_SUCCESS;
}
[1] https://en.wikiversity.org/wiki/Damm_algorithm
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 17:36:11 UTC, Nestor wrote:
Hi,
I was trying to port C code from the article in Wikiversity [1]
to D, but I'm not sure this implementation is the most
efficient way to do it in D, so suggestions to optimize it are
welcome:
import std.stdio;
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
(tmpdigit - '0') * 10];
return tmpdigit;
}
enum {
EXIT_SUCCESS = 0,
EXIT_FAILURE = 1,
}
int main(string[] args) {
scope(failure) {
writeln("Invalid arguments. You must pass a number.");
return EXIT_FAILURE;
}
assert(args.length == 2);
char digit = checkDigit(args[1]);
if(digit == '0') writefln("%s is a valid number.", args[1]);
else {
writefln("%s is not a valid number (but it would be,
appending digit %s).",
args[1], digit);
}
return EXIT_SUCCESS;
}
[1] https://en.wikiversity.org/wiki/Damm_algorithm
I can't see how to make it more efficient than it is (aside from
finding a better algorithm which isn't a language concern). You
are using C style and will get the same machine code as if you'd
written it with C. The only difference would be boundchecking
that you can disable by using the -boundscheck=off compiler
switch.
By the way note that you can use the -profile switch to see where
the bottlenecks are, in your case writting the result to stdout
is *by far* what takes the most time (and I'm quite certain the
bottleneck is the same in the C version for it's a fact that I/O
is slow).
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 17:36:11 UTC, Nestor wrote:
I was trying to port C code from the article in Wikiversity [1]
to D, but I'm not sure this implementation is the most
efficient way to do it in D, so suggestions to optimize it are
welcome:
import std.stdio;
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
(tmpdigit - '0') * 10];
return tmpdigit;
}
Well one thing is you can probably reduce them from chars to just
bytes, instead of having to subtract you can instead add at the
end. Although unless you're working with a VERY large input you
won't see a difference.
Actually since you're also multiplying by 10, you can incorporate
that in the table too... (although a mixin might be better for
the conversion than by hand)
static immutable char[] QG10Matrix = [
00,30,10,70,50,90,80,60,40,20,
70,00,90,20,10,50,40,80,60,30,
40,20,00,60,80,70,10,30,50,90,
10,70,50,00,90,80,30,40,20,60,
60,10,20,30,00,40,50,90,70,80,
30,60,70,40,20,00,90,50,80,10,
50,80,60,90,70,20,00,10,30,40,
80,90,40,50,30,60,20,00,10,70,
90,40,30,80,60,10,70,20,00,50,
20,50,80,10,40,30,60,70,90,00];
char checkDigit(string str) {
char tmpdigit = 0;
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
tmpdigit];
return (tmpdigit/10) + '0';
}
Re: Strange behaviour of rdmd vs. dmd concerning main function
Dne 9.2.2017 v 17:20 berni via Digitalmars-d-learn napsal(a): dmd only compiles in the files you actually pass to it. rdmd will try to find the required files automatically. Since you didn't pass the file with the function to dmd, it knows it exists, but leaves it out of the final link (it assumes it might come from a library or something). That's why you see the error. Ah ok, I understand. So calling with "dmd Special/special.d Common/common.d" works. But when I compile common.d to common.o (with dmd -c common.d) and remove common.d after that is there still a possibility to link? The obvious doesn't work: $> dmd Special/special.d Common/common.o Special/special.d(4): Error: module common is in file 'common.d' which cannot be read import path[0] = /usr/include/dmd/phobos import path[1] = /usr/include/dmd/druntime/import Thats ok too, you still need to add -ICommon dmd -ICommon Special/special.d Common/common.o
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 18:34:30 UTC, Era Scarecrow wrote:
On Thursday, 9 February 2017 at 17:36:11 UTC, Nestor wrote:
I was trying to port C code from the article in Wikiversity
[1] to D, but I'm not sure this implementation is the most
efficient way to do it in D, so suggestions to optimize it are
welcome:
import std.stdio;
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
(tmpdigit - '0') * 10];
return tmpdigit;
}
Well one thing is you can probably reduce them from chars to
just bytes, instead of having to subtract you can instead add
at the end. Although unless you're working with a VERY large
input you won't see a difference.
Actually since you're also multiplying by 10, you can
incorporate that in the table too... (although a mixin might be
better for the conversion than by hand)
static immutable char[] QG10Matrix = [
00,30,10,70,50,90,80,60,40,20,
70,00,90,20,10,50,40,80,60,30,
40,20,00,60,80,70,10,30,50,90,
10,70,50,00,90,80,30,40,20,60,
60,10,20,30,00,40,50,90,70,80,
30,60,70,40,20,00,90,50,80,10,
50,80,60,90,70,20,00,10,30,40,
80,90,40,50,30,60,20,00,10,70,
90,40,30,80,60,10,70,20,00,50,
20,50,80,10,40,30,60,70,90,00];
char checkDigit(string str) {
char tmpdigit = 0;
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
tmpdigit];
return (tmpdigit/10) + '0';
}
OK I changed the approach using a multidimensional array for the
matrix so I could ditch arithmetic operations altogether, but
curiously after measuring a few thousand runs of both
implementations through avgtime, I see no noticeable difference.
Why?
import std.stdio;
static immutable ubyte[][] QG10Matrix = [
[0,3,1,7,5,9,8,6,4,2],[7,0,9,2,1,5,4,8,6,3],
[4,2,0,6,8,7,1,3,5,9],[1,7,5,0,9,8,3,4,2,6],
[6,1,2,3,0,4,5,9,7,8],[3,6,7,4,2,0,9,5,8,1],
[5,8,6,9,7,2,0,1,3,4],[8,9,4,5,3,6,2,0,1,7],
[9,4,3,8,6,1,7,2,0,5],[2,5,8,1,4,3,6,7,9,0],
];
static int charToInt(char chr) {
scope(failure) return -1;
return cast(int)(chr - '0');
}
ubyte checkDigit(string str) {
ubyte tmpdigit;
foreach(chr; str) tmpdigit =
QG10Matrix[tmpdigit][charToInt(chr)];
return tmpdigit;
}
enum {
EXIT_SUCCESS = 0,
EXIT_FAILURE = 1,
}
int main(string[] args) {
scope(failure) {
writeln("Invalid arguments. You must pass a number.");
return EXIT_FAILURE;
}
assert(args.length == 2);
ubyte digit = checkDigit(args[1]);
if(digit == 0) writefln("%s is a valid number.", args[1]);
else {
writefln("%s is not a valid number (but it would be,
appending digit %s).",
args[1], digit);
}
return EXIT_SUCCESS;
}
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 18:34:30 UTC, Era Scarecrow wrote: ... Actually since you're also multiplying by 10, you can incorporate that in the table too... I forgot to comment that what is multiplied by ten is not the value but the starting position in the array (a way to emulate a matrix), but anyway see my comments in previous post.
Re: Strange behaviour of rdmd vs. dmd concerning main function
On Thursday, 9 February 2017 at 19:10:55 UTC, Daniel Kozak wrote: Dne 9.2.2017 v 17:20 berni via Digitalmars-d-learn napsal(a): [...] Ah ok, I understand. So calling with "dmd Special/special.d Common/common.d" works. But when I compile common.d to common.o (with dmd -c common.d) and remove common.d after that is there still a possibility to link? The obvious doesn't work: [...] Thats ok too, you still need to add -ICommon dmd -ICommon Special/special.d Common/common.o Doesn't work here: $> dmd -ICommon Special/special.d Common/common.o Special/special.d(4): Error: module common is in file 'common.d' which cannot be read import path[0] = Common import path[1] = /usr/include/dmd/phobos import path[2] = /usr/include/dmd/druntime/import
Getting familiar with std.process
I haven't used std.process before and am trying to play around
with it.
In the code below, the first file writes some text to an output
file. My goal is to be able to read what is written to that file
without creating the file itself. I'm not sure it's possible, but
the second file is my attempt.
The second file compiles the first (for the sake of simplicity),
then creates a pipe with program it compiles, piping the result
to stdout, then it saves the result to an output string, which I
then print (or potentially manipulate in some other way in the
future). But it doesn't print anything (why I'm posting here).
I think the issue is that create_file doesn't write to stdout, it
writes to file. Other than reading the file and then deleting it,
I don't know what else to try.
//create_file.d
import std.stdio : toFile;
void main()
{
toFile("test", "output.txt");
}
//read_file.d
//read_file.d
import std.process;
import std.stdio;
void main()
{
auto command1 = execute(["dmd", "create_file.d"]);
//auto command2 = execute(["create_file.exe"]);
auto pipes = pipeProcess("create_file.exe", Redirect.stdout);
scope(exit) wait(pipes.pid);
foreach (line; pipes.stdout.byLine) writeln(line);
}
Re: Can this implementation of Damm algorithm be optimized?
Dne 9.2.2017 v 20:39 Nestor via Digitalmars-d-learn napsal(a):
On Thursday, 9 February 2017 at 18:34:30 UTC, Era Scarecrow wrote:
On Thursday, 9 February 2017 at 17:36:11 UTC, Nestor wrote:
I was trying to port C code from the article in Wikiversity [1] to
D, but I'm not sure this implementation is the most efficient way to
do it in D, so suggestions to optimize it are welcome:
import std.stdio;
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') + (tmpdigit -
'0') * 10];
return tmpdigit;
}
Well one thing is you can probably reduce them from chars to just
bytes, instead of having to subtract you can instead add at the end.
Although unless you're working with a VERY large input you won't see
a difference.
Actually since you're also multiplying by 10, you can incorporate
that in the table too... (although a mixin might be better for the
conversion than by hand)
static immutable char[] QG10Matrix = [
00,30,10,70,50,90,80,60,40,20,
70,00,90,20,10,50,40,80,60,30,
40,20,00,60,80,70,10,30,50,90,
10,70,50,00,90,80,30,40,20,60,
60,10,20,30,00,40,50,90,70,80,
30,60,70,40,20,00,90,50,80,10,
50,80,60,90,70,20,00,10,30,40,
80,90,40,50,30,60,20,00,10,70,
90,40,30,80,60,10,70,20,00,50,
20,50,80,10,40,30,60,70,90,00];
char checkDigit(string str) {
char tmpdigit = 0;
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
tmpdigit];
return (tmpdigit/10) + '0';
}
OK I changed the approach using a multidimensional array for the
matrix so I could ditch arithmetic operations altogether, but
curiously after measuring a few thousand runs of both implementations
through avgtime, I see no noticeable difference. Why?
import std.stdio;
static immutable ubyte[][] QG10Matrix = [
[0,3,1,7,5,9,8,6,4,2],[7,0,9,2,1,5,4,8,6,3],
[4,2,0,6,8,7,1,3,5,9],[1,7,5,0,9,8,3,4,2,6],
[6,1,2,3,0,4,5,9,7,8],[3,6,7,4,2,0,9,5,8,1],
[5,8,6,9,7,2,0,1,3,4],[8,9,4,5,3,6,2,0,1,7],
[9,4,3,8,6,1,7,2,0,5],[2,5,8,1,4,3,6,7,9,0],
];
static int charToInt(char chr) {
scope(failure) return -1;
return cast(int)(chr - '0');
}
ubyte checkDigit(string str) {
ubyte tmpdigit;
foreach(chr; str) tmpdigit = QG10Matrix[tmpdigit][charToInt(chr)];
return tmpdigit;
}
enum {
EXIT_SUCCESS = 0,
EXIT_FAILURE = 1,
}
int main(string[] args) {
scope(failure) {
writeln("Invalid arguments. You must pass a number.");
return EXIT_FAILURE;
}
assert(args.length == 2);
ubyte digit = checkDigit(args[1]);
if(digit == 0) writefln("%s is a valid number.", args[1]);
else {
writefln("%s is not a valid number (but it would be, appending
digit %s).",
args[1], digit);
}
return EXIT_SUCCESS;
}
Maybe you can try use static array instead of dynamic
static immutable ubyte[10][10] QG10Matrix = ...
Re: Strange behaviour of rdmd vs. dmd concerning main function
Dne 9.2.2017 v 21:10 berni via Digitalmars-d-learn napsal(a): On Thursday, 9 February 2017 at 19:10:55 UTC, Daniel Kozak wrote: Dne 9.2.2017 v 17:20 berni via Digitalmars-d-learn napsal(a): [...] Ah ok, I understand. So calling with "dmd Special/special.d Common/common.d" works. But when I compile common.d to common.o (with dmd -c common.d) and remove common.d after that is there still a possibility to link? The obvious doesn't work: [...] Thats ok too, you still need to add -ICommon dmd -ICommon Special/special.d Common/common.o Doesn't work here: $> dmd -ICommon Special/special.d Common/common.o Special/special.d(4): Error: module common is in file 'common.d' which cannot be read import path[0] = Common import path[1] = /usr/include/dmd/phobos import path[2] = /usr/include/dmd/druntime/import Which version of dmd you have? It should work, you probably do something wrong, I have use your example files so do you change them somehow?
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 19:39:49 UTC, Nestor wrote:
On Thursday, 9 February 2017 at 18:34:30 UTC, Era Scarecrow
wrote:
On Thursday, 9 February 2017 at 17:36:11 UTC, Nestor wrote:
I was trying to port C code from the article in Wikiversity
[1] to D, but I'm not sure this implementation is the most
efficient way to do it in D, so suggestions to optimize it
are welcome:
import std.stdio;
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
(tmpdigit - '0') * 10];
return tmpdigit;
}
Well one thing is you can probably reduce them from chars to
just bytes, instead of having to subtract you can instead add
at the end. Although unless you're working with a VERY large
input you won't see a difference.
Actually since you're also multiplying by 10, you can
incorporate that in the table too... (although a mixin might
be better for the conversion than by hand)
static immutable char[] QG10Matrix = [
00,30,10,70,50,90,80,60,40,20,
70,00,90,20,10,50,40,80,60,30,
40,20,00,60,80,70,10,30,50,90,
10,70,50,00,90,80,30,40,20,60,
60,10,20,30,00,40,50,90,70,80,
30,60,70,40,20,00,90,50,80,10,
50,80,60,90,70,20,00,10,30,40,
80,90,40,50,30,60,20,00,10,70,
90,40,30,80,60,10,70,20,00,50,
20,50,80,10,40,30,60,70,90,00];
char checkDigit(string str) {
char tmpdigit = 0;
foreach(chr; str) tmpdigit = QG10Matrix[(chr - '0') +
tmpdigit];
return (tmpdigit/10) + '0';
}
OK I changed the approach using a multidimensional array for
the matrix so I could ditch arithmetic operations altogether,
but curiously after measuring a few thousand runs of both
implementations through avgtime, I see no noticeable
difference. Why?
import std.stdio;
static immutable ubyte[][] QG10Matrix = [
[0,3,1,7,5,9,8,6,4,2],[7,0,9,2,1,5,4,8,6,3],
[4,2,0,6,8,7,1,3,5,9],[1,7,5,0,9,8,3,4,2,6],
[6,1,2,3,0,4,5,9,7,8],[3,6,7,4,2,0,9,5,8,1],
[5,8,6,9,7,2,0,1,3,4],[8,9,4,5,3,6,2,0,1,7],
[9,4,3,8,6,1,7,2,0,5],[2,5,8,1,4,3,6,7,9,0],
];
static int charToInt(char chr) {
scope(failure) return -1;
return cast(int)(chr - '0');
}
ubyte checkDigit(string str) {
ubyte tmpdigit;
foreach(chr; str) tmpdigit =
QG10Matrix[tmpdigit][charToInt(chr)];
return tmpdigit;
}
enum {
EXIT_SUCCESS = 0,
EXIT_FAILURE = 1,
}
int main(string[] args) {
scope(failure) {
writeln("Invalid arguments. You must pass a number.");
return EXIT_FAILURE;
}
assert(args.length == 2);
ubyte digit = checkDigit(args[1]);
if(digit == 0) writefln("%s is a valid number.", args[1]);
else {
writefln("%s is not a valid number (but it would be,
appending digit %s).",
args[1], digit);
}
return EXIT_SUCCESS;
}
The question is, why do you expect it to be noticably faster? On
modern hardware it the optimizations are such that so little a
change in code is very hard to link to a difference in running
time. If you really want to show one the following code does the
benchmark the right way (ie: using a very long input, tens of
thousands of runs, and avoiding I/O and load time of the program
to compare the bare function implementations):
import std.conv;
import std.stdio;
import std.range;
import std.array;
import std.algorithm;
string testcase;
static immutable char[] QG10MatrixOne =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
char checkDigitOne(string str) {
char tmpdigit = 0;
foreach(chr; str) tmpdigit = QG10MatrixOne[(chr - '0') +
(tmpdigit - '0') * 10];
return tmpdigit;
}
void testCheckDigitOne() {
checkDigitTwo(testcase);
}
static immutable ubyte[][] QG10MatrixTwo = [
[0,3,1,7,5,9,8,6,4,2],[7,0,9,2,1,5,4,8,6,3],
[4,2,0,6,8,7,1,3,5,9],[1,7,5,0,9,8,3,4,2,6],
[6,1,2,3,0,4,5,9,7,8],[3,6,7,4,2,0,9,5,8,1],
[5,8,6,9,7,2,0,1,3,4],[8,9,4,5,3,6,2,0,1,7],
[9,4,3,8,6,1,7,2,0,5],[2,5,8,1,4,3,6,7,9,0],
];
static int charToInt(char chr) {
scope(failure) return -1;
return cast(int)(chr - '0');
}
ubyte checkDigitTwo(string str) {
ubyte tmpdigit;
foreach(chr; str) {
tmpdigit = QG10MatrixTwo[tmpdigit][charToInt(chr)];
}
return tmpdigit;
}
void testCheckDigitTwo() {
checkDigitTwo(testcase);
}
void main(string[] args) {
testcase =
iota(10).cycle.take(10).map!(to!string).array.join;
import std.datetime: benchmark;
benchmark!(testCheckDigitOne,
testCheckDigitTwo)(1).each!writeln;
}
Yet on my machine I have the following times (note that the time
itself depends on my hardware, what's important is the
difference):
TickDuration(15785255852)
TickDuration(15784826803)
So it seems that the first version is slower than the second one,
but by so little that it's hard to link to the actual
implementation. If anything it shows the futility of su
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 20:46:06 UTC, Daniel Kozak wrote:
Maybe you can try use static array instead of dynamic
static immutable ubyte[10][10] QG10Matrix = ...
I shaved it to this to discard unneccessary time-consuming
functions:
static immutable ubyte[10][10] QG10Matrix = [
[0,3,1,7,5,9,8,6,4,2],[7,0,9,2,1,5,4,8,6,3],
[4,2,0,6,8,7,1,3,5,9],[1,7,5,0,9,8,3,4,2,6],
[6,1,2,3,0,4,5,9,7,8],[3,6,7,4,2,0,9,5,8,1],
[5,8,6,9,7,2,0,1,3,4],[8,9,4,5,3,6,2,0,1,7],
[9,4,3,8,6,1,7,2,0,5],[2,5,8,1,4,3,6,7,9,0],
];
static immutable string number =
"0123456789012345678901234567890123456789012345678901234567890123456789";
static int charToInt(char chr) {
return ((chr >= '0') || (chr <= '9')) ? cast(int)(chr - '0') :
-1;
}
ubyte checkDigit(string str) {
ubyte tmpdigit;
foreach(chr; str) tmpdigit =
QG10Matrix[tmpdigit][charToInt(chr)];
return tmpdigit;
}
void main() {
auto digit = checkDigit(number);
}
I even tried making checkDigit static, but surprisingly this
increased average execution time by 1ms.
Anyway, the previopus version (modified to benefit from a little
optimization too) is almost as performant, even though it
includes multiplication and sum:
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
static immutable string number =
"0123456789012345678901234567890123456789012345678901234567890123456789";
static int charToInt(char chr) {
return ((chr >= '0') || (chr <= '9')) ? cast(int)(chr - '0') :
-1;
}
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit =
QG10Matrix[charToInt(chr) + (charToInt(tmpdigit) * 10)];
return tmpdigit;
}
void main() {
auto digit = checkDigit(number);
}
I compiled both with -inline -noboundscheck -release and the
multidimensional array version does perform 1ms faster for a
couple hundred runs, but I expected the difference to be much
more noticeable.
Any idea of what might be happening here?
Re: Getting familiar with std.process
On 02/09/2017 12:44 PM, jmh530 wrote: > I think the issue is that create_file doesn't write to stdout, it writes > to file. Correct. Pipe works by piping the standard input/output streams. > Other than reading the file and then deleting it, I don't know > what else to try. create_file must write to its stdout. Ali
Re: Can this implementation of Damm algorithm be optimized?
Dne 9.2.2017 v 22:29 Nestor via Digitalmars-d-learn napsal(a):
On Thursday, 9 February 2017 at 20:46:06 UTC, Daniel Kozak wrote:
Maybe you can try use static array instead of dynamic
static immutable ubyte[10][10] QG10Matrix = ...
I shaved it to this to discard unneccessary time-consuming functions:
static immutable ubyte[10][10] QG10Matrix = [
[0,3,1,7,5,9,8,6,4,2],[7,0,9,2,1,5,4,8,6,3],
[4,2,0,6,8,7,1,3,5,9],[1,7,5,0,9,8,3,4,2,6],
[6,1,2,3,0,4,5,9,7,8],[3,6,7,4,2,0,9,5,8,1],
[5,8,6,9,7,2,0,1,3,4],[8,9,4,5,3,6,2,0,1,7],
[9,4,3,8,6,1,7,2,0,5],[2,5,8,1,4,3,6,7,9,0],
];
static immutable string number =
"0123456789012345678901234567890123456789012345678901234567890123456789";
static int charToInt(char chr) {
return ((chr >= '0') || (chr <= '9')) ? cast(int)(chr - '0') : -1;
}
ubyte checkDigit(string str) {
ubyte tmpdigit;
foreach(chr; str) tmpdigit = QG10Matrix[tmpdigit][charToInt(chr)];
return tmpdigit;
}
void main() {
auto digit = checkDigit(number);
}
I even tried making checkDigit static, but surprisingly this increased
average execution time by 1ms.
Anyway, the previopus version (modified to benefit from a little
optimization too) is almost as performant, even though it includes
multiplication and sum:
static immutable char[] QG10Matrix =
"03175986427092154863420687135917509834266123045978" ~
"36742095815869720134894536201794386172052581436790";
static immutable string number =
"0123456789012345678901234567890123456789012345678901234567890123456789";
static int charToInt(char chr) {
return ((chr >= '0') || (chr <= '9')) ? cast(int)(chr - '0') : -1;
}
char checkDigit(string str) {
char tmpdigit = '0';
foreach(chr; str) tmpdigit =
QG10Matrix[charToInt(chr) + (charToInt(tmpdigit) * 10)];
return tmpdigit;
}
void main() {
auto digit = checkDigit(number);
}
I compiled both with -inline -noboundscheck -release and the
multidimensional array version does perform 1ms faster for a couple
hundred runs, but I expected the difference to be much more noticeable.
Any idea of what might be happening here?
Did you try it with different backends? llvm (ldc), gcc(gdc)?
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 21:43:08 UTC, Daniel Kozak wrote: Any idea of what might be happening here? Did you try it with different backends? llvm (ldc), gcc(gdc)? Not really, just standard dmd. I tried running each algoritm a few times through avgtime using different digit lengths (upto 10,000, my PC won't handle much more) and different amount of repetitions, and the results aren't consistent, some times one algorithm is marginally faster, sometimes the other. Apparently the compiler is capable of optimizing the unidimensional array version. Thank you all nevertheless for the suggestions.
Re: Can this implementation of Damm algorithm be optimized?
On 02/09/2017 02:04 PM, Nestor wrote: > I tried running each algoritm a few times through avgtime using > different digit lengths avgtime profiles the whole process, right? It measures everything that is involved in that little program. At least OS starting the program, D runtime initializing, program interacting with the console, etc. Since you're comparing algorithms, you need to take everything else out of the equation. A better approach is to use std.datetime.benchmark, which would repeat just the algorithm that you're interested in. Ali
Re: Can this implementation of Damm algorithm be optimized?
Dne 9.2.2017 v 23:08 Ali Çehreli via Digitalmars-d-learn napsal(a): On 02/09/2017 02:04 PM, Nestor wrote: > I tried running each algoritm a few times through avgtime using > different digit lengths avgtime profiles the whole process, right? It measures everything that is involved in that little program. At least OS starting the program, D runtime initializing, program interacting with the console, etc. Since you're comparing algorithms, you need to take everything else out of the equation. A better approach is to use std.datetime.benchmark, which would repeat just the algorithm that you're interested in. Ali Yes this is important, and for example when you used shared libphobos it makes really big difference
Re: Can this implementation of Damm algorithm be optimized?
On Thursday, 9 February 2017 at 19:39:49 UTC, Nestor wrote: OK I changed the approach using a multidimensional array for the matrix so I could ditch arithmetic operations altogether, but curiously after measuring a few thousand runs of both implementations through avgtime, I see no noticeable difference. Why? Truthfully, because you'll need tens of millions or hundreds of millions in length to determine if it makes a difference and how much. Addition, subtraction and simple memory lookups take very little time, and since the entire array (100 bytes) fits in the cache, it is going to perform very very very well regardless if you can optimize it further. If you tested this on a much slower system, say an 8bit 6502 the differences would be far more pronounced, but not that much different. Since the algorithm is more or less O(n) optimizing it won't make many differences. It's possible you could get a speedup by making them ints instead of chars, since then it might not have a penalty for the 'address not divisible by 4' that applies (which is more for ARM architectures and less for x86). Other optimizations could be to make it multiple levels, taking the basic 100 elements and expanding them 2-3 levels deep in a lookup and having it do it in more or less a single operation. (100 bytes for 1 level, 10,000 for 2 levels, 1,000,000 for 3 levels, 100,000,000 for 4 levels, etc), but the steps of converting it to the array lookup won't give you that much gain, although fewer memory lookups but none of them will be cached, so any advantage from that is probably lost. Although if you bump up the size to 16x10 instead of 10x10, you could use a shift instead of *10 which will make that slightly faster (there will be unused empty padded spots) In theory if you avoid the memory lookup at all, you could gain some amount of speed, depending on how it searches a manual table, although using a switch-case and a mixin to do all the details it feels like it wouldn't give you any gain... Division operations are the slowest operations you can do, but otherwise most instructions run really fast. Unless you're trying to get it smaller (fewer bytes for the call) or shaving for speed by instruction cycle counting (like on the 6502) i doubt you'll get much benefit.
Re: Strange behaviour of rdmd vs. dmd concerning main function
On Thursday, 9 February 2017 at 16:20:29 UTC, berni wrote: dmd only compiles in the files you actually pass to it. rdmd will try to find the required files automatically. Since you didn't pass the file with the function to dmd, it knows it exists, but leaves it out of the final link (it assumes it might come from a library or something). That's why you see the error. Ah ok, I understand. So calling with "dmd Special/special.d Common/common.d" works. But when I compile common.d to common.o (with dmd -c common.d) and remove common.d after that is there still a possibility to link? The obvious doesn't work: $> dmd Special/special.d Common/common.o Special/special.d(4): Error: module common is in file 'common.d' which cannot be read import path[0] = /usr/include/dmd/phobos import path[1] = /usr/include/dmd/druntime/import This is not a linker error. It's a compiler error. You need common.d for the import, so the compiler can know which symbols are available. For any modules you import, you need either the original source file (.d) or a D interface (header) file (.di) which contains the type & function declarations and any template implementations you need.
Re: Getting familiar with std.process
On Thursday, 9 February 2017 at 21:36:46 UTC, Ali Çehreli wrote: On 02/09/2017 12:44 PM, jmh530 wrote: > I think the issue is that create_file doesn't write to stdout, it writes > to file. Correct. Pipe works by piping the standard input/output streams. > Other than reading the file and then deleting it, I don't know > what else to try. create_file must write to its stdout. Ali If only it were that easy. It's not my program writing to file, it's some one I didn't write. This was just sort of a beta test. I might need to look into the program a bit more.
