On 10/23/17 16:47, Nathan S. wrote:
On Monday, 23 October 2017 at 22:22:55 UTC, Adam Wilson wrote:
Additionally, MSFT/C# fully recognizes that the benefits of
Async/Await have never been and never were intended to be for
performance. Async/Await trades raw performance for an ability to
handle a truly massive number of simultaneous tasks.
Could you clarify this? Do you mean it's not supposed to have better
performance for small numbers of tasks, but there is supposed to be some
high threshold of tasks/second at which either throughput or latency is
better?
It's pretty complicated. In general, for a small number of tasks, it
will take longer to execute those task with Async than without (Async
has an overhead, about 10ms IIRC). So each individual call to an
awaitable function will take longer than a call to a blocking function.
In fact, MSFT recommends that if you are reasonably certain that most of
the time it will take less than about 10ms to just use the blocking
methods, as they have less total overhead.
The main benefit of Async is in throughput. It allows the physical CPU
to handle many incoming requests. So while each individual request may
take longer, the overall utilization of the CPU is much higher.
Async, has different benefits and drawbacks depending on where it's
applied. For example, in UI apps, it allows the main program thread to
keep responding to system events while waiting for long-wait IO (the
thread is not suspended).
Some background that may help understanding is that in blocking IO, what
is really happening underneath your blocking call is that the runtime is
creating a callback and then calling the OS's thread suspend method
(e.g. ThreadSuspend in Windows), then when the callback is called, the
thread is resumed and the data passed to into it via the callback. This
means that the calling thread cannot execute anything while it's
waiting. This is why UX apps appear to freeze when using blocking IO
calls. The reason this is done is because the application has no way to
say "I'm going to put this task off to the side and keep executing". The
thread does not know to start some other processing while waiting.
Async allows the app to put that task to the aside and do something
else. At a low level the process in .NET it does this by:
1. Serializing the stack frame of the currently executing method to the
heap (yes, Async is a GC feature, just like lambdas) at an await.
2. Pulling the next completed task from the heap.
3. Rebuilding the stack frame for that method on any available thread.
4. Continue execution of that stack.
Obviously this makes a lot of sense for UI apps, where blocking the main
thread can be disastrous, but why internet applications are inherently
multi-threaded, so why do we care?
The answer is thread context switching. A context switch is the most
expensive common CPU operation by an order of magnitude, ranging from
2k-1m cycles. Whereas on modern CPUs a main RAM read is 100-150 cycles
and a NUMA different socket read is 300-500 cycles. In .NET the
TaskScheduler creates a predetermined number of threads (one per core
IIRC) and begins scheduling tasks on those threads. Remembering that
each task is really just a block on the heap, in the worst case that
will take 500 cycles. Whereas if we had to switch to a different thread,
it could be up to 1m cycles. That is a noticeable difference.
Context switches are painfully expensive but in the traditional model it
was all we had. Task based systems allow us to circumvent the task
switch. But they're cumbersome to use without compiler support. For
example, the Task Parallel Library was added in .NET 4.0 which includes
Task and Task<T> and ALL of the constructs that are used in Async/Await,
however, it was not until .NET 4.5 and the arrival of the Async/Await
keywords (and the compiler lowerings that they enabled) that people
started using Tasks in any significant way.
(Source for access times:
http://ithare.com/wp-content/uploads/part101_infographics_v08.png)
--
Adam Wilson
IRC: LightBender
import quiet.dlang.dev;
