AFAIK the only way to do non-blocking file IO is by doing it asynchronously via a non-blocking queue like the disruptor.
For networking IO the channel/selector stuff is interesting when you need to scale up to tens of thousands of connections and you get beyond the place where you can have a dedicated thread per connection. As Ralph mentioned, many experts in this area warn that this is much, much harder to get right and only gives you better _scalability_, not better throughput performance. That said, what I find interesting about NIO is the ByteBuffer API. It has a more powerful API than streams, but gives a single API that still abstracts over whether you are writing to a slab of heap memory, off-heap memory, a memory-mapped file, or even directly to a device. I like the fact that it combines a stream-like API (non-indexed get/put methods) with a random-access (the indexed get/put methods) API, and the various other goodies like bulk get/put, views, compacting, slicing etc. The price you pay is slightly higher complexity. When someone passes you a ByteBuffer and you want to read from it, the question is always, has this buffer already been flipped(), or do I need to flip() it myself, and this can be a source of confusion and bugs. A ByteBuffer API may not be applicable everywhere, but when used judiciously it can help avoid unnecessary copying and improve performance. I'm thinking it may be possible to have zero copy by using a ByteBuffer API for things like binary loggers ( https://issues.apache.org/jira/browse/LOG4J2-506 ), in combination with the memory-mapped file appender. Perhaps a bit of a narrow field though... :-) Remko On Sun, Apr 27, 2014 at 7:20 PM, Ralph Goers <[email protected]> wrote: > OK, but again, why? The articles I've sen say it is harder to implement > and slower. I might be able to see using it for sockets but I have a hard > time seeing what benefit there would be for file appenders. > > Ralph > > On Apr 27, 2014, at 12:16 AM, Matt Sicker <[email protected]> wrote: > > The neat feature is non-blocking IO. We could use the Channel API for > non-blocking IO in appenders. > > > On 26 April 2014 23:35, Ralph Goers <[email protected]> wrote: > >> 1. Why would it be a neat feature? >> 2. The FileManager uses NIO to lock the file if it was requested. >> 3. Many of the Appenders use Streams. I happen to like that as it allows >> them to share common code. >> >> Ralph >> >> On Apr 26, 2014, at 6:48 PM, Matt Sicker <[email protected]> wrote: >> >> I see that the current FileAppender/FileManager classes (and parents) all >> work via streams. Am I missing something, or are there no appenders using >> NIO? This could be a neat feature for 2.1. >> >> -- >> Matt Sicker <[email protected]> >> >> >> > > > -- > Matt Sicker <[email protected]> > >
