Hi Pavel, Thanks for asking about this. I was meant to send an update to the list earlier but never got around to it.
As you probably noticed from the history of this thread, there was a deafening silence in response to it. We decided that core-libs-dev might not have been the best place to start this effort. So we've taken it to Eclipse MicroProfile, and the intention is to incubate it there, and then once we have something that we're reasonably happy with, look at contributing that to the JDK. That strategy has so far gone really well. The project lives here: https://github.com/eclipse/microprofile-reactive-streams We have a specification: http://download.eclipse.org/microprofile/microprofile-reactive-1.0-M1/microprofile-reactive-streams-operators-spec.html A TCK (see the TCK section of the spec for info). Comprehensive API docs: http://download.eclipse.org/microprofile/microprofile-reactive-1.0-M1/apidocs/ And all this has been produced by a team of around 10 contributors from five different companies (Lightbend, RedHat, IBM, Payara and Tomitribe), with all meetings recorded and minuted: https://docs.google.com/document/d/1UKBBNFn-CeMih3lNUGvpShcL2aTpnvmJs9LQ8qT9OI0/edit# We are about to release a 1.0-RC1, and are planning to have a 1.0 out in time for Oracle Code One, where there will be some presentations on it. My plan was, once 1.0 was released, to resume discussions on this list about whether this could be contributed to the JDK, possibly as an incubating module. Are there any questions in particular that you have about it? If you're interested, we have an open meeting every Tuesday, 1100 UTC, that you're welcome to join (actually next week we're expecting some other Oracle devs to join from the team that produced Helidon). A link to the calendar event for this meeting, with links to the Zoom meeting and minutes, can be found here: https://calendar.google.com/calendar/event?eid=NDk3ZjQxMjkyZGh1N2lrZnU2dTk0Y29laWtfMjAxODA5MjVUMTEwMDAwWiBnYm5iYzM3M2dhNDBuMHR2Ymw4OG5rYzNyNEBn&ctz=Etc/GMT Cheers, James On Tue, 28 Aug 2018 at 00:33, Pavel Rappo <pavel.ra...@oracle.com> wrote: > Hi James, > > What's the status of the project? Has it been abandoned? > > -Pavel > > > On 22 Mar 2018, at 00:09, James Roper <ja...@lightbend.com> wrote: > > > > Hi all, > > > > We've created a reference implementation. It's been done in such a way > that > > implementation of new features (stages) is quite straight forward, there > is > > an abstraction that does the hard work of handling concurrency, the > > reactive streams spec conformance, and managing demand and buffering, and > > error handling, and then individual stages (eg, map, filter, flatMap) are > > implemented using a very easy to use API (note, this API/abstraction is > all > > private, internal to the reference implementation). Rudimentary tests on > > performance show that it's not terrible compared to other reactive > streams > > implementations, with a number of clear optimization paths identified > > should we decide that's necessary. I believe this proposal is now close > to > > complete - the remaining work is: > > > > * Decide what scope beyond JDK8 streams it should support - while this > > decision is not trivial, the amount of work required to actually add > these > > to the API and implement in the reference implementation is trivial. > > * Fill out the TCK with more rigorous verification. > > * Create some rigorous benchmarks. > > > > I'm not sure what should be done next. I have talked to a number of > people > > who are either involved in, or are writing APIs that use Reactive Streams > > in private, and interest seems high. Also, there is general consensus in > > public discussions in the Jakarta EE/MicroProfile communities that an API > > like this would be very valuable in the JDK. The API of course could be > > done in Jakarta EE instead, but given that Reactive Streams is part of > and > > used by the JDK, and given that the JDK8 Streams API is part of the JDK, > > Jakarta EE would seem an odd place to put this library - it essentially > > would mean that to make effective use of JDK libraries that use Reactive > > Streams (eg HTTP client, possibly java.sql2 aka ADBA), you need to use > > Jakarta EE (or some third party streaming library). > > > > So unless there's any major feedback coming here on this list, I'd like > to > > put this forward as a JEP. > > > > Regards, > > > > James > > > > On 15 March 2018 at 12:24, James Roper <ja...@lightbend.com> wrote: > > > >> Hi all, > >> > >> An update on this. We've now filled out the API with feature parity with > >> the JDK8 Streams API - for operators that make sense in Reactive > Streams. > >> We've provided example implementations of the API backed by both Akka > >> Streams and rxjava, showing that it can be widely implemented. The TCK > >> still needs some work, but covers the major features, and > comprehensively > >> covers all publishers/subscribers with the Reactive Streams TCK (so > >> comprehensive that we actually found two Reactive Streams TCK > violations in > >> Akka Streams with this, and a couple in rxjava too). > >> > >> There are two major areas of work left to get something that would be > >> ready to be a candidate for a final API. The first is to produce a zero > >> dependency reference implementation of the API. This is what I plan on > >> starting on next. > >> > >> The second is to decide what additional operators and generators the API > >> should provide. So far, the scope has been mostly limited to a subset of > >> the JDK8 Streams scope, only a few additional API pieces have been > created, > >> such as a few variations on flatMap (one that supports CompletionStage, > and > >> one that supports Iterable). There are a number of other features for > >> consideration to provide basic necessary functionality for this API, > here's > >> some examples off the top of my head (some of these can already be > >> implemented in terms of other stages already provided): > >> > >> * A cancelled subscriber (useful for cleaning up hot publishers) > >> * An ignoring subscriber (useful when you do the actual work in a > previous > >> stage of the graph, such as mapping to completion stages) > >> * Error handling subscribers and/or processors > >> * Termination listening subscribers and/or processors > >> * A processor that wraps a subscriber and publisher > >> * The ability to merge streams - so far only concat is provided, and all > >> flatMaps are essentially concatenations, merge variants may be useful > >> (though introduce a lot of complexity, such as specifying breadth) > >> * The ability to split streams into sub streams - a use case for this is > >> in parsing a stream that contains potentially large sub streams, like > >> parsing a multipart/form-data body > >> * Batching of elements in a stream, based on predicate, influenced by > >> backpressure or based on a scheduled tick > >> * Scheduled features such as emitting ticks, rate limiting, etc > >> * The ability to control buffering and asynchronous boundaries within a > >> graph > >> * Naming of stages for debug/error reporting/monitoring purposes > >> > >> Not all of the above may be absolutely necessary, but should be > >> considered, and there may be other features as well that would be > useful to > >> consider. > >> > >> Please visit the repo and any feedback would be much appreciated: > >> > >> https://github.com/lightbend/reactive-streams-utils > >> > >> Regards, > >> > >> James > >> > >> On 8 March 2018 at 03:59, Brian Goetz <brian.go...@oracle.com> wrote: > >> > >>> To answer the questions at the bottom: the next step is to start > working > >>> on this and get folks excited about contributing. There's plenty of > time > >>> for process later, but filing a JEP or creating a project shouldn't be > a > >>> barrier to innovating. > >>> > >>> > >>> On 2/28/2018 10:33 PM, James Roper wrote: > >>> > >>>> Hi all, > >>>> > >>>> We've put together a simple proposal for this. Please read the README > for > >>>> an introduction to this proposal. > >>>> > >>>> https://github.com/lightbend/reactive-streams-utils > >>>> > >>>> Regards, > >>>> > >>>> James > >>>> > >>>> On 22 February 2018 at 11:47, James Roper <ja...@lightbend.com> > wrote: > >>>> > >>>> Hi all, > >>>>> > >>>>> This is an email to give people a heads up that we'd like to look at > >>>>> creating an API, in the same vein as the JDK8 Streams API, for > building > >>>>> reactive streams (a la JDK9 juc.Flow). Our goals for this are: > >>>>> > >>>>> * To fill a gap in the JDK where if a developer wants to do even the > >>>>> simplest of things with a JDK9 juc.Flow, such as map or filter, they > >>>>> need > >>>>> to bring in a third party library that implements that. > >>>>> * To produce an API that can build Publishers, Subscribers, > Processors, > >>>>> and complete graphs, for the purposes of consuming APIs that use > >>>>> reactive > >>>>> streams (for example, JDK9 Http Client). > >>>>> * To produce an API that aligns closely with ju.stream.Stream, using > it > >>>>> for inspiration for naming, scope, general API shape, and other > aspects. > >>>>> The purpose of this goal is to ensure familiarity of Java developers > >>>>> with > >>>>> the new API, and to limit the number of concepts Java developers > need to > >>>>> understand to do the different types of streaming offered by the JDK. > >>>>> * To produce an API that can be implemented by multiple providers > >>>>> (including an RI in the JDK itself), using the ServiceLoader > mechanism > >>>>> to > >>>>> provide and load a default implementation (while allowing custom > >>>>> implementations to be manually provided). There are a lot of concerns > >>>>> that > >>>>> each different streams implementation provides and implements, beyond > >>>>> streaming, for example monitoring/tracing, concurrency modelling, > >>>>> buffering > >>>>> strategies, performance aspects of the streams handling including > >>>>> fusing, > >>>>> and context (eg thread local) propagation. This will allow libraries > to > >>>>> use > >>>>> and provide contracts based on this API without depending on a > >>>>> particular > >>>>> implementation, and allows developers to select the implementation > that > >>>>> meets their needs. > >>>>> > >>>>> Non goals: > >>>>> > >>>>> * To produce a kitchen sink of utilities for working with reactive > >>>>> streams. There already exist a number of reactive streams > >>>>> implementations > >>>>> that seek to meet this goal (eg, Akka Streams, Reactor, RxJava), and > >>>>> once > >>>>> you go past the basics (map, filter, collect), and start dealing with > >>>>> things like fan in/out, cycles, restarting, etc, the different > >>>>> approaches > >>>>> to solving this start to vary greatly. The JDK should provide enough > to > >>>>> be > >>>>> useful for typical every day streaming use cases, with developers > being > >>>>> able to select a third party library for anything more advanced. > >>>>> > >>>>> We will update this list when we have something ready for public > review. > >>>>> This probably won't be far off. Our hope is that we can propose this > as > >>>>> a > >>>>> JEP. > >>>>> > >>>>> Regards, > >>>>> > >>>>> James > >>>>> > >>>>> -- > >>>>> *James Roper* > >>>>> *Senior Octonaut* > >>>>> > >>>>> Lightbend <https://www.lightbend.com/> – Build reactive apps! > >>>>> Twitter: @jroper <https://twitter.com/jroper> > >>>>> > >>>>> > >>>> > >>>> > >>> > >> > >> > >> -- > >> *James Roper* > >> *Senior Octonaut* > >> > >> Lightbend <https://www.lightbend.com/> – Build reactive apps! > >> Twitter: @jroper <https://twitter.com/jroper> > >> > > > > > > > > -- > > *James Roper* > > *Senior Octonaut* > > > > Lightbend <https://www.lightbend.com/> – Build reactive apps! > > Twitter: @jroper <https://twitter.com/jroper> > > -- *James Roper* *Senior Developer, Office of the CTO* Lightbend <https://www.lightbend.com/> – Build reactive apps! Twitter: @jroper <https://twitter.com/jroper>
