Re: HTOD
On Tuesday, 22 August 2017 at 17:15:27 UTC, Walter Bright wrote: You're right about htod, and it's on me. It's built out of the DMC++ front end. I haven't gotten around yet to releasing it as open source. We can discuss possible ways of implementing htod. Instead, I'd rather discuss how we can make D more approachable and attractive to people thinking of picking up the language. In that respect, as far as htod goes, I think it should be removed from the site (it could still be available online). D can't have an official command line utility that doesn't work. Small steps.
Re: Community Rant
On Tuesday, 22 August 2017 at 15:48:17 UTC, Kagamin wrote: Other possibilities can be dstep or cpp2d from visuald project. Though don't know if the latter can work on linux. So I guess someone should pick one and put it on the site. And make sure the source code is available. Having a link to a broken unusable utility on the main language website looks bad, to say the least.
Re: Community Rant
On Tuesday, 22 August 2017 at 15:24:54 UTC, ixid wrote: On Tuesday, 22 August 2017 at 15:14:33 UTC, Jonathan Shamir wrote: various. Out of interest did you pick up D before or after joining the start up? If before did you introduce D to them or were they already using it? I work at weka.io. I learned D at weka, same as most of our workers (including the founders that looked for a powerful system programming language).
Community Rant
https://dlang.org/htod.html I click download and get an exe! And in the bugs section: No linux version. I'll start with the productive part. If anyone can point me out to the sources of htod I would love to compile for linux + osx. Any task seems more attractive to me than manually converting a 1000 line header to D. I'm a D lover and advocate. I actually get a salary writing D code for a cutting-edge startup. But lets be honest. If I was just interested to learn about this "modern system programming language" that is C++ done right, I would dismiss D very quickly. We need to get together as a community and rethink your priorities, because with problems like this we're making it very hard for newcomers to trust in this very poorly adapted language. Programming tools used by day to day programmers should be a priority. Because everyone expects valgrind to work. The standard library should be a priority. It's far from complete (hopefully my company will contribute in this respect in the near future). The DUB package repository is horrible! More often than not, the packages are so poorly written I end up just writing my own implementation. Adding the ability to "rate" packages would go a long way in improving the situation. I understand hacking the frontend is way more interesting to most of the community. But if we don't find the time to improve on our visibility and language maturity, D will never get the attention it deserves. P.S. I don't know you guys (except Ali and Andrei which I had the honor to meet). I don't follow the forums. I'm sure you often speak about these topics here. So - if I offended anyone know it's not personal (I don't know who you are). I just want to share my impressions and experience as an actual day to day D user.
std.process.execute performance without Config.inheritFDs
Hey,
This code is from std.process:
if (!(config & Config.inheritFDs))
{
import core.sys.posix.sys.resource;
rlimit r;
getrlimit(RLIMIT_NOFILE, &r);
foreach (i; 3 .. cast(int) r.rlim_cur) close(i);
}
This is a close-loop to make sure no fds are leaked before
execve'ing, and can be disabled using Config.inheritFDs.
Here's a program I used to profile:
void main() {
StopWatch sw;
import core.sys.posix.sys.resource;
rlimit r;
errnoEnforce(getrlimit(RLIMIT_NOFILE, &r) == 0);
writefln("The rlimit is %d (hard: %s)", r.rlim_cur,
r.rlim_max);
//r.rlim_cur = 1024 * 1024;
//errnoEnforce(setrlimit(RLIMIT_NOFILE, &r) == 0);
sw.start();
foreach (_; 0 .. 100) {
executeShell("echo hello", null,
std.process.Config.inheritFDs);
}
sw.stop();
writefln("Total time: %s", sw.peek().to!Duration.to!string);
}
These are the results I got:
// On a typical linux machine, without inheritFDs
The rlimit is 524288 (hard: 1048576)
Total time: 3 secs, 875 ms, 759 μs, and 7 hnsecs
// with inheritFDs
The rlimit is 524288 (hard: 1048576)
Total time: 80 ms, 549 μs, and 2 hnsecs
// osx, without inheritFDs
The rlimit is 7168 (hard: 9223372036854775807)
Total time: 476 ms, 483 μs, and 5 hnsecs
// with inheritFDs
The rlimit is 7168 (hard: 9223372036854775807)
Total time: 352 ms, 637 μs, and 7 hnsecs
So obviously this becomes a problem if the rlimit is high.
A few suggestions:
1. Make inheritFDs the default, since most people aren't aware of
the performance cost. Also most programs will run just fine if a
few fds are leaked.
1.1. Or, by default, close up to min(rlim_cur, 1024) - this
should be enough for most processes. (If I'm not mistaken, that's
what they do in python). Also this change won't break existing
code.
2. All phobos fds should be opened with O_CLOEXEC, as intended.
This eliminates the need to close all the fds when execve'ing.
3. Optimization for linux - use /proc to check which fds are
actually opened, and close them, instead of making thousands of
syscalls!
Re: auto decoding rant
On Thursday, 15 June 2017 at 15:50:42 UTC, Seb wrote: On Thursday, 15 June 2017 at 15:47:54 UTC, Jonathan Shamir wrote: Also note I can't cast to char[] in compile time? What's the reason for that? Have a look at: https://dlang.org/phobos/std_utf.html#byCodeUnit https://dlang.org/phobos/std_utf.html#byChar https://dlang.org/phobos/std_string.html#.representation Perfect, thanks!
auto decoding rant
I see this is a recurring rant (I apologize if this is a
repeating topic, I'm new to the forums). Here's an example of
something that should be simple in D but isn't:
enum string PATTERN = "abcd";
immutable char[10] repeatingPattern =
PATTERN.cycle().takeExactly(10).array();
The fact that front(string) returns a dchar makes some really
simple (and common) use-cases practically impossible.
Also note I can't cast to char[] in compile time? What's the
reason for that?
I would recommend adding something like this to phobos (and in
all fairness, it should have been the other way around):
auto noDecode(T)(T[] str) if (isNarrowString!(T[])) {
static struct NoDecode {
private T[] str;
@property ref T front() {
assert (!this.empty);
return str[0];
}
@property bool empty() {
return (str.empty);
}
void popFront() {
if (!this.empty) {
str = str[1 .. $];
}
}
NoDecode save() {
return this;
}
}
return NoDecode(str);
}
alias this and shadowing
I think the following example is legitimate code that should work:
struct S1 {
void foo() {
writeln("S1.foo()");
}
void foo(ulong x) {
writefln("S1.foo(%d)", x);
}
}
struct S2 {
S1 s;
alias s this;
void foo(string str) {
writeln("S2.foo(%s)", str);
}
}
void main() {
S2 s;
s.foo("hahaha");
s.foo(100); // XXX this doesn't compile, since there's no
S2.foo that accepts an int
}
To quote Andrei, if it looks like it should work, it should.
(Also something about turtles).
My original code did something a bit different, just to show
another example of why this behavior looks fundamentally broken
to me:
struct S2 {
S1 s;
alias s this;
@disable void foo(ulong); // calling foo(ulong) doesn't make
sense in the context of S2
}
Now, calling S2.foo() won't compile, since S2 only knows one
prototype for foo which accepts a ulong.
Re: [WEKA] Calling fork() and D runtime
OK well, seems like calling _exit() (instead of exit()) did the trick. Thanks for the help!
[WEKA] Calling fork() and D runtime
Hey,
This is my first time writing in the D forums!
I have an application written in D that runs as a linux daemon
(some python service script is in charge of running and
daemonizing it).
This "agent" works similar to docker - a service that accepts
commands and runs in the background, and starts our application
container. The container is itself a daemon (ppid = 1) with
unshared namespaces etc.
So, normally, implementing such an application would look
something like:
1. Main "agent" process runs fork() to create a child process
(child_1). All memory is copy-on-write.
2. Child_1 malloc()s a stack, and calls clone() to create yet
another child (child_2), which will eventually become the
container pid 1.
3. Child_2 initializes the container (mounts, unshare, chroot,
etc) then eventually exec()s into the container init process.
4. child_1 exit()s, which causes child_2 to become a daemon.
5. The agent main process should wait() on the forked pid since
it's impolite to leave zombies (I do this in a thread).
The problem I encounter is with the forked process (child_1).
Here is the code I wrote handling the fork() (Note: this
functionality should really be provided by core.threads or
something, for unix environments).
```private void deferToForkProcess(int delegate() entryPoint,
Timeout timeout = Timeout.infinite) {
import core.runtime : Runtime;
import core.sys.posix.unistd : fork, pid_t;
import core.sys.posix.sys.wait;
import core.stdc.stdlib : exit;
int rc = theReactor.deferToThread({
pid_t pid = fork();
errnoEnforce(pid >= 0, "Fork failed");
// child process
if (pid == 0) {
try {
int rc = entryPoint();
exit(rc);
} catch (Throwable ex) {
try {
LOG_ERROR(ex.toString);
} catch (Throwable) {}
exit(1);
}
//assert(false);
}
// parent - wait for child to exit
int status = 0;
do {
errnoEnforce(waitpid(pid, &status, 0) != -1, "Waitpid
failed");
} while (!WIFEXITED(status));
int rc = WEXITSTATUS(status);
return rc;
}, timeout);
enforce(rc == 0, "Child process failed (rc = %d)".format(rc));
}
```
entryPoint() returns 0, but the exit(0) raises an
OutOfMemoryError:
```0x4e6472
exit
??:0
0x4e6428
__run_exit_handlers
??:0
0x4df976
__libc_csu_fini
??:0
0x40327e
ldc.register_dso
crtstuff.c:0
0x4caee4
_d_dso_registry
??:0
0x4ccdba
_D2rt4util9container6common8xreallocFNbNiPvmZPv
??:0
0x4b873d
onOutOfMemoryError
??:0```
I tried to call Runtime.initialize() and Runtime.terminate()
surrounding the call to entryPoint(), but this didn't help. I
suspect calling initialize() was a no-op since the forked process
shares (copy-on-write) the VM space with it's parent, that
already initialized the runtime. (Note: confirmed, initialize()
returns true indicating it was already inited).
What is the correct way to handle fork() with the D runtime?
