I've run into a problem with Async fileio. On Posix,
the follwing interface is required:
proc faio_write(q: job_queue,
fd: fd_t, len: &int, buf: address, eof: &bool
)
{
faio_rw(q, fd, len, buf, eof, false); // write
}
The 'q' there is a worker_fifo thing. With the current Felix async
I/O design, the user is required to create a thread pool to run
synchronous jobs asynchronously. For example standard fileio
is synchronous, but it can be made asynchronous by doing the
io operation in a thread and notifying when the operation has
finished.
In the old async system, there was a worker_fifo created by
the driver (whether required or not), and, every spawn_pthread
would duplicate the Felix event loop, including all the
async I/O threads.
In the new system, the USER must create these threads:
for timing, the user must create an alarm clock and
pass it as an argument to the sleep function.
Because duplicating Felix does NOT duplicate the thread_frame_t
object .. and because of the explicit requirement to pass
an argument which is statically type checked, we can obtain
sharing of these queues/workers/threads, and, not need to
have one running if the user program doesn't actually need it.
Some operations don't require a worker, for example socket IO
on posix uses the kernel notification mechanism instead.
Similarly, on Windows, IO completion ports provide notification
for file I/O .. so Windows doesn't need any worker queue.
However in principle ALL these functions should require a queue,
because it's an implementation detail that Felix happens to use
kernel operations for some async I/O ops and not others.
The problem is seen here:
instance OByteStream[fd_t]
{
proc write(s: fd_t, len: &int, buf: address, eof: &bool)
#if POSIX
{ faio_write(s, len, buf, eof); }
#elif WIN32
{ WriteFile(s, len, buf, eof); }
#endif
}
Well this just can't work, because there's no queue argument.
Just to make sure we understand: the worker queue is a neutral
synchronisation technique: there's a pool of threads which
pull jobs of the queue, run them, signal they're finished,
and then run the next one. The pool size is user controlled.
The use of a thread pool isn't really appropriate, since you
can hang up waiting for jobs to complete and fail to start
new jobs that might complete before any of the existing ones.
Separate, specialised, queues are probably better: the timer
subsystem already uses a specialised timer queue which dispatches
timer waits in O(1) time using the OS notification services
or a single polling loop.
Nevertheless, the job queue is a reasonable approximation
for starters, and has the virtual it can 'asynchronise'
anything.
So the PROBLEM here is that the stream interfaces don't
accept a job queue, and they need to .. which means using
streams is going to require a queue be created by the user
even if on their platform none is required for any operations.
AND it means all the code using the stream interface is broken
and will have to be fixed.
To make it seem more 'intuitive' that a queue really is
required, just consider fileio requests on your local
disk as opposed to an NFS mounted volume. NFS can be
very slow .. it makes sense to have TWO queues,
one for the local disk and one for NFS, to that the fact
thread A is waiting on NFS doesn't stop thread B doing a local
disk operation.
Remember Felix async I/O does NOT block a pthread .. it
only blocks the requesting fthread/fibre .. so even if you
have a single pthread program you STILL may want multiple
job queues.
A better design might use a priority queue .. but it's hard
to know how to set priorities (as usual, integers suck, we
want an arbitrary ordering here).
--
John Skaller <skaller at users dot sf dot net>
Felix, successor to C++: http://felix.sf.net
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Felix-language mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/felix-language
