Hello,
SmokingRope wrote:
Hi, I'm poking through the pieces of the System.Core/System.IO.Pipes
code looking to build out anonymous pipes. I have at least a basic
working Unix implementation now. Though i imagine Win32 and Named
pipes have been broken (if they ever worked) in the process, and there
is undoubtedly some cleanup work required. I'd like to get some
feedback from people more knowledgeable than myself about mono at this
point so i can try and wrap this project up into a pull request.
That's great! Thanks for taking care of this.
I'm skipping some parts of your questions and writing for some parts
that I can answer...
My initial test case is this example from MSDN:
http://msdn.microsoft.com/en-us/library/bb546102.aspx
Handle Inheritance
-------------------------
Presently, Process.Start() arbitrarily closes all handles in the
fork()'d process, except stdin / stdout / stderr. There exists a todo
comment in this same code (io-layer/processes.c) pointing out that
(when inherit_handles is TRUE, which it is) that something should be
done differently than when not...
if (/inherit_handles/ != TRUE) {
/* FIXME: do something here */
}
Without a far more expansive knowledge of mono i can't make a very
well informed decision about the liberties i can take in addressing this.
The cleanest solution might be to use FD_CLOEXEC on all the existing
fd's and (at least in process.Start case) the handles that should be
cleaned up would be closed automatically / inherited automatically.
Are there compatibility reasons this cannot be done? Has there been
any consideration for this that could be drawn upon?
>From information i do have, it seems like these pipe handles could
also be flagged as 'inheritable' in the internal handle registry and
then after fork() mono could leave those open, while closing
everything else.
In addition to this basic test case, i'm not currently aware of how
anonymous pipes should behave when you invoke Process.Start() several
times with an anonymous pipe. Without using FD_CLOEXEC it would be
difficult and messy to (in a 'execd' child process) to identify all
the pipe handles at startup and register them in the mono handle
registry as inheritable again (so that Process.Start in that child
process would pass these handles on again to its children). If
anonymous pipes are supposed to support this, it lends further support
for the FD_CLOEXEC solution.
API Methods Difficult To Implement In Unix
-------------------------
There doesn't seem to be a clear specification for the anonymous pipe
feature and i've so far had to make some little assumptions /
inferences based on the documentation on MSDN. That said, the posix
anonymous pipe standard lacks a few features which are pertinent to
the API for anonymous pipe streams. In particular there are two
conflicting features:
* WaitForPipeDrain
<http://msdn.microsoft.com/en-us/library/system.io.pipes.pipestream.waitforpipedrain.aspx>()
defined on the AnonymousPipeStream requires some ability to peak into
an anonymous pipe and wait for the pipe's buffer to become empty
* Write / Read / etc... is supposed to throw an IOException if
the other end of the pipe stream has been closed (i have interpreted
the wording 'pipe is broken' from the msdn docs to mean this, although
there is a chance i am mis-interpreting this too)
These two features conflict when trying to implement them in unix
through minor hacks / workarounds.
I can implement WaitForPipeDrain() by leaving the read end of the pipe
handle open in the writer process, and calling poll(3) on the read end
repeatedly until the POLL_IN flag is gone.
I can detect that the pipe is broken by closing the read end on the
writer, (and the write end on the reader) and calling poll(3),
throwing this "pipe broken" error when a POLLERR /. POLLHUP is issued.
One of these two flags will be set if all handles to the other end of
the pipe have been closed.
In order to implement both of these conflicting features i actually
need to create two pipes (one where i can leave the read end open on
the writer process, and one where i can close it). This is not very
nice in the case that system resources might be at a premium, however
i haven't been able to devise any other way to get both features
either. Would there be any major objection to implementing it like
this (which as i have noted seems to be the only way)?
I would say that opening two pipes is of less concern. If people need
very efficient named pipes then they should rather use platform-specific
stuff.
Using two separate pipes like this, the GetClientHandleAsString()
method is going to be significantly different from the win32 version.
When the client is supposed to be the writer, the client handle as
string must contain three separate fds. When the client is the reader
two fd's must be passed. I don't see any reason that this would bother
anybody, but if it's relevant let me know!
That method can return totally different string than what .NET returns.
It's weird that if an application totally depends on certain format of
the return values of it and I believe we can ignore them.
Compatibility
-------------------------
I can create pipes and manipulate them without adding any new library
references but i do depend on using the poll(3) function more heavily
than it seems to be presently (it's only exposed for Network Sockets
presently). Are there any portability / compatibility considerations
with poll(3) that might need to be addressed?
I guess Mono.Posix.dll has a lot more bound API that you want to use (it
is already referenced by System.Core.dll for desktop CLR_PROFILE in
System.Core/Makefile).
If that still needs to touch our runtime (io-layer) then it is of our
concern yes, as we are building runtime not only for desktop. Yet we
have couple of defs (PLATFORM_ANDROID etc.), so as long as things can be
easily conditioned out it would work.
Versioning
-------------------------
I'm not familiar with how versioning between .NET 1/2/3/etc... is
being handled. Anonymous pipes are introduced in .NET 3.5. Do these
classes need to be hidden to older versions, etc... Maybe there is a
wiki page i haven't found discussing some of this?
Our least supported Framework API is 2.0 runtime, which covers 2.0, 3.0,
3.5 and 3.5 SP1 API. So, you don't need to add any condition. For 4.0
specific types and members, use "#if NET_4_0" for all of them (you can
hide them by changing them as internal for !NET_4_0).
I have added several new internal calls to the MonoIO utility,
specifically for poll(3) and pipe handle manipulation. Does adding
internal calls require any versioning considerations?
Testing
-------------------------
In order to write a test suite for anonymous pipes i need to start new
processes and i will need to use some sort of IPC mechanism to
communicate certain errors back to the test process. This seems like a
rather unique challenge compared to most of the testing needed for mono.
Has any of this multi-process unit testing been done before in mono,
or are there any considerations that might be relevant?
Not everything has to be in our existing NUnit test style. We have a
couple of standalone tests (e.g. in System.Web.Services or System.Xml).
Probably you can create your own test runner to deal with multiple
processes.
Please do not hesitate to show us your ongoing changes. I'm sure that
almost no one cared this API (at least no one contributed to the
implementation), so you're most likely the one at best knowledge here :)
And with code people can talk better.
Atsushi Eno
_______________________________________________
Mono-devel-list mailing list
Mono-devel-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-devel-list
