Hi Travis, Thanks for the kind words :) Hopefully once network support gets done and it's the default that all the work pays off :)
Cheers, Tamar Sent from my Mobile On Sat, Jul 18, 2020, 06:38 Travis Whitaker <pi.boy.tra...@gmail.com> wrote: > Hello Tamar, > > Just wanted to send a quick note expressing my gratitude for these > improvements! My first professional work in Haskell targeted Windows > platforms, and I'm sure there are many others out there who appreciate > the tireless work you've done to keep GHC's Windows support chugging > along. I've been kicking the tires on your WinIO branch for a few > months and this is absolutely a game-changer for Windows support. > > Thanks! > > Travis > > On Fri, Jul 17, 2020 at 8:06 AM Phyx <loneti...@gmail.com> wrote: > > > > Hi All, > > > > In case you've missed it, about 150 or so commits were committed to > master > > yesterday. These commits add WinIO (Windows I/O) to GHC. This is a new > I/O > > manager that is designed for the native Windows I/O subsystem instead of > > relying on the broken posix-ish compatibility layer that MIO used. > > > > This is one of 3 big patches I have been working on for years now.. > > > > So before I continue on why WinIO was made I'll add a TL;DR; > > > > WinIO adds an internal API break compared to previous GHC releases. > That is > > the internal code was modified to support a completely asynchronous I/O > system. > > > > What this means is that we have to keep track of the file pointer offset > which > > previously was done by the C runtime. This is because in async I/O you > cannot > > assume the offset to be at any given location. > > > > What does this mean for you? Very little. If you did not use internal > GHC I/O code. > > In particular if you haven't used Buffer, BufferIO and RawIO. If you > have you will > > to explicitly add support for GHC 8.12+. > > > > Because FDs are a Unix concept and don't behave as you would expect on > Windows, the > > new I/O manager also uses HANDLE instead of FD. This means that any > library that has > > used the internal GHC Fd type won't work with WinIO. Luckily the number > of libraries > > that have seems quite low. If you can please stick to the external > Handle interface > > for I/O functions. > > > > The boot libraries have been updated, and in particular process > *requires* the version > > that is shipped with GHC. Please respect the version bounds here! I > will be writing > > a migration guide for those that need to migrate code. The amount of > work is usually > > trivial as Base provides shims to do most of the common things you would > have used Fd for. > > > > Also if I may make a plea to GHC developers.. Do not add non-trivial > implementations > > in the external exposed modules (e.g. System.xxx, Data.xxx) but rather > add them to internal > > modules (GHC.xxx) and re-export them from the external modules. This > allows us to avoid > > import cycles inside the internal modules :) > > > > -- > > > > So why WinIO? Over the years a number of hard to fix issues popped up on > Windows, including > > proper Unicode console I/O, cooked inputs, ability to cancel I/O > requests. This also allows libraries like Brick to work on Windows without > re-inventing the wheel or have to hide their I/O from the I/O manager. > > > > In order to attempt to do some of these with MIO layer upon layers of > hacks were added. This means that things sometimes worked.., but when it > didn't was rather unpredictable. Some of the issues were simply unfixable > with MIO. I will be making some posts about how WinIO works (and also > archiving them on the wiki don't worry :)) but for now some highlights: > > > > WinIO is 3 years of work, First started by Joey Hess, then picked up by > Mikhail Glushenkov before landing at my feet. While the majority has been > rewritten their work did provide a great jumping off point so thanks! Also > thanks to Ben and AndreasK for helping me get it over the line.. As you can > imagine I was exhausted by this point :). > > > > Some stats: ~8000 new lines and ~1100 removed ones spread over 130+ > commits (sorry this was the smallest we could get it while not losing some > historical context) and with over 153 files changed not counting the > changes to boot libraries. > > > > It Fixes #18307, #17035, #16917, #15366, #14530, #13516, #13396, #13359, > #12873, #12869, #11394, #10542, #10484, #10477, #9940, #7593, #7353, #5797, > #5305, #4471, #3937, #3081, #12117, #2408, #10956, #2189 > > (but only on native windows consoles, so no msys shells) and #806 which > is 14 years old! > > > > WinIO is a dynamic choice, so you can switch between I/O managers using > the RTS flag --io-manager=[native|posix]. > > > > On non-Windows native is the same as posix. > > > > The chosen Async interface for this implementation is using Completion > Ports. > > > > The I/O manager uses a new interface added in Windows Vista called > GetQueuedCompletionStatusEx which allows us to service multiple request > interrupts in one go. > > > > Some highlights: > > > > * Drops Windows Vista support > > Vista is out of extended support as of 2017. The new minimum is > Windows 7. This allows us to use much more efficient OS provided > abstractions. > > > > * Replace Events and Monitor locks with much faster and efficient > Conditional Variables and SlimReaderWriterLocks. > > * Change GHC's Buffer and I/O structs to support asynchronous operation > by not relying on the OS managing File Offset. > > * Implement a new command line flag +RTS --io-manager=[native|posix] to > control which I/O manager is used. > > * Implement a new Console I/O interface supporting much faster > reads/writes and unicode output correctly. Also supports things like > cooked input etc. > > * In new I/O manager if the user still has their code-page set to OEM, > then we use UTF-8 by default. This allows Unicode to work correctly out of > the box. > > * Add Atomic Exchange PrimOp and implement Atomic Ptr exchanges. > > * Flush event logs eagerly as to not rely on finalizers running. > > * A lot of refactoring and more use of hsc2hs to share constants > > * Control aborts Ctrl+C should be a bit more reliable. > > * Add a new IOPort primitive that should be only used for these I/O > operations. Essentially an IOPort is based on an MVar with the following > major > > differences: > > - Does not allow multiple pending writes. If the port is full a second > write is just discarded. > > - There is no deadlock avoidance guarantee. If you block on an IOPort > and your Haskell application does not have any work left to do the whole > application is > > stalled. In the threaded RTS we just continue idling, in the > non-threaded rts the scheduler is blocked. > > > > * Support various optimizations in the Windows I/O manager such as > skipping I/O Completion if the request finished synchronously etc. > > * The I/O manager is now agnostic to the handle type. i.e. There is no > socket specific code in the manager. This is now all pushed to the network > library. Completely de-coupling these. > > * Unified threaded and non-threaded I/O code. The only major difference > is where event loop is driven from and that the non-threaded rts will > always use a single OS thread to service requests. We cannot use more as > there are no rts locks to make concurrent modifications safe. > > > > Cheers, > > Tamar > > _______________________________________________ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs