On Thursday, 25 September 2025 at 07:24:00 UTC, Sönke Ludwig
wrote:
Am 23.09.25 um 17:35 schrieb Dmitry Olshansky:
On Monday, 22 September 2025 at 11:14:17 UTC, Sönke Ludwig
wrote:
Am 22.09.25 um 09:49 schrieb Dmitry Olshansky:
On Friday, 19 September 2025 at 17:37:36 UTC, Sönke Ludwig
wrote:
So you don't support timeouts when waiting for an event at
all? Otherwise I don't see why a separate API would be
required, this should be implementable with plain Posix
APIs within vibe-core-lite itself.
Photon's API is the syscall interface. So to wait on an
event you just call poll.
Behind the scenes it will just wait on the right fd to
change state.
Now vibe-core-light wants something like read(buffer,
timeout) which is not syscall API but maybe added. But since
I'm going to add new API I'd rather have something
consistent and sane not just a bunch of adhoc functions to
satisfy vibe.d interface.
Why can't you then use poll() to for example implement
`ManualEvent` with timeout and interrupt support? And
shouldn't recv() with timeout be implementable the same way,
poll with timeout and only read when ready?
Yes, recv with timeout is basically poll+recv. The problem is
that then I need to support interrupts in poll. Nothing really
changed.
As far as manual event goes I've implemented that with custom
cond var and mutex. That mutex is not interruptible as it's
backed by semaphore on slow path in a form of eventfd.
I might create custom mutex that is interruptible I guess but
the notion of interrupts would have to be introduced to
photon. I do not really like it.
I'd probably create an additional event FD per thread used to
signal interruption and also pass that to any poll() that is
used for interruptible wait.
poll could be made interruptible w/o any additions it's really a
yield + waiting for events. I could implement interruptible
things by simply waking fiber up with special flag
passed (it has such to determine which event waked us up anyway).
The problem is, I do not see how to provide adequate interface
for this functionality.
I have an idea though - use EINTR or maybe devise my own code for
interrupts, then I could interrupt at my syscall interface level.
Next is the question of how do I throw from a nothrow context.
Here I told you that I would need a separate APIs for
interruptible things - the ones that allow interrupts. *This* is
something I do not look forward to.
I think we have a misunderstanding of what vibe.d is supposed
to be. It seems like you are only focused on the web/server
role, while to me vibe-core is a general-purpose I/O and
concurrency system with no particular specialization in
server tasks. With that view, your statement to me sounds
like "Clearly D is not meant to do multi- threading, since
main() is only running in a single thread".
The defaults are what is important. Go defaults to
multi-threading for instance.
D defaults to multi-threading because TLS by default is
certainly a mark of multi-threaded environment.
std.concurrency defaults to new thread per spawn, again this
tells me it's about multithreading. I intend to support
multi-threading by default. I understand that we view this
issue differently.
But you are comparing different defaults here. With plain D,
you also have to import either `core.thread` or
`std.concurrency`/`std.paralellism` to do any multi-threaded
work. The same is true for vibe-core. What you propose would be
more comparable to having foreach() operate like
parallelForeach(), with far-reaching consequences.
If we are just talking about naming - runTask/runWorkerTask vs.
go/goOnSameThread - that is of course debatable, but in that
case I think it's blown very much out of proportion to take
that as the basis to claim "it's meant to be used
single-threaded".
So runTask is assumed to run on the same core while runWorkerTask
to be run on any available core? Didn't occur to me. I thought
worker pool is for blocking tasks, as there is such a pool in
photon. I can just switch runTask to goOnSameThread to maximize
compatibility with vibed.
Channels tame the complexity. Yes, channels could get more
expansive in multi-threaded scenario but we already agreed
that it's not CPU bound.
If you have code that does a lot of these things, this just
degrades code readability for absolutely no practical gain,
though.
I humbly disagree. I'd take explicit channels over global TLS
variables any day.
This doesn't make sense, in the original vibe-core, you can
simply choose between spawning in the same thread or in
"any" thread. `shared`/`immutable` is correctly enforced in
the latter case to avoid unintended data sharing.
I have go and goOnSameThread. Guess which is the encouraged
option.
Does go() enforce proper use of shared/immutable when passing
data to the scheduled "go routine"?
It goes with the same API as we have for threads - a delegate,
so sharing becomes user's responsibility. I may add function +
args for better handling of resources passed to the lambda.
That means that this is completely un`@safe` - C++ level memory
safety. IMO this is an unacceptable default for web
applications.
Yeah, I'm not in the @safe world mostly. But as I said to make it
more upstreamable I will switch the defaults, so that
vibe-core-light provides the same guarantees as regular vibe-core
does.
malloc() will also always be a bottleneck with the right
load. Just the n times larger amount of virtual address space
required may start to become an issue for memory heavy
applications. But even if ignore that, ruling out using the
existing GC doesn't sound like a good idea to me.
The existing GC is basically 20+ years old, ofc we need better
GC and
thread cached allocation solves contention in multi-threaded
environments.
Alternative memory allocator is doing great on 320 core
machines. I cannot tell you which allocator that is or what
exactly these servers are. Though even jemalloc does okayish.
And the fact is that, even with relatively mild GC use, a web
application will not scale properly with many cores.
Only partially agree, Java's GC handles load just fine and
runs faster than vibe.d(-light). It does allocations on its
serving code path.
I was just talking about the current D GC here. Once we have a
better implementation, this can very well become a much weaker
argument!
However, speaking more generally, the other arguments for
preferring to scale using processes still stand, and even with
a better GC I would still argue that leading library users to
do multi-threaded request handling is not necessarily the best
default (of course it still *can* be for some applications).
I'm betting more on the threaded approach, but we are just
different. See also my reply on the numbers - processes are only
about 1-2% faster (and the noise is easily in 0.5% range) once
the GC bottleneck is handled that is.
Anyway, the main point from my side is just that the semantics
of what *is* in vibe-core-light should really match the
corresponding functions in vibe-core. Apart from that, I was
just telling you that your impression of it being intended to
be used single-threaded is not right, which doesn't mean that
the presentation shouldn't probably emphasize the
multi-threaded functionality and multi-threaded request
processing more.
Given the number of potential expectations from the user side it
seems I need to update vibe-core-light to use goOnSameThread for
runTask. I do not like it how I need to do extra work to launch a
multi-threaded server though which is something that got me
started on the whole "defaults argument".
