Yes, I agree.
+1s on both these lines:
*Badly designed APIs can get more complicated when combined with lambdas*
*Lambdas and functional style can greatly improve the readability if used
properly*.
On Mon, Jan 2, 2017 at 9:34 PM, Arun Mahadevan <ar...@apache.org> wrote:
> Its more about getting used to and judiciously deciding when to use
> functional v/s procedural approach. Badly designed APIs can get more
> complicated when combined with lambdas, but overall what I observe is
> lambdas and functional style can greatly improve the readability (with many
> other benefits) if used properly.
>
> A simple word count pipeline like below can get too verbose when expressed
> without lambdas.
>
> Stream<String> stream = …
> stream
> .flatMap(s -> Arrays.asList(s.split(" ")))
> .mapToPair(w -> Pair.of(w, 1))
> .countByKey()
> .filter(x -> x.getSecond() >= 5)
> .print();
>
> *Without lambda*
>
> stream
> .flatMap(new FlatMapFunction<String, String>() {
> @Override
> public Iterable<String> apply(String s) {
> return Arrays.asList(s.split(" "));
> }
> })
> .mapToPair(new PairFunction<String, String, Integer>() {
> @Override
> public Pair<String, Integer> apply(String w) {
> return Pair.of(w, 1);
> }
> })
> .countByKey()
> .filter(new Predicate<Pair<String, Long>>() {
> @Override
> public boolean test(Pair<String, Long> x) {
> return x.getSecond() >= 5;
> }
> })
> .print();
>
>
> Thanks,
> Arun
>
> On 1/3/17, 10:21 AM, "S G" <sg.online.em...@gmail.com> wrote:
>
> >One bad usage I have found is missing types for lambda arguments.
> >When the types for arguments are not provided in the code itself, github
> or
> >vi-editor browsing of such code becomes impossible.
> >
> >You must have an IDE like Eclipse to infer the types for such cases and
> >that becomes a big limitation to browse such code. I have not looked into
> >the storm code-base to find such occurrences but the above is just a
> >general observation from the lambda code usages.
> >
> >
> >Secondly, lambdas tend to generate bad stack-traces on exceptions.
> >
> >
> >And lastly, functional programming is not natural to many Java developers.
> >https://zeroturnaround.com/rebellabs/how-to-avoid-
> ruining-your-world-with-lambdas-in-java-8/
> >hints at some of these problems. Java code is usually very easy to
> >understand. But once you throw in more than a single level of lambdas,
> >things tend to become more and more confusing for several developers.
> (This
> >last point reflects the thoughts from some of my fellow developers, not
> >representative of all the devs)
> >
> >
> >Thanks
> >SG
> >
> >
> >On Mon, Jan 2, 2017 at 8:26 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
> >
> >> Hi S G,
> >>
> >> I'd be happy if you could elaborate your opinion. Did you found bad
> usages
> >> from master branch of Storm code?
> >>
> >> Regarding comfortable of lambda, IMHO, I don't think many users are
> >> unfamiliar with lambda, since they should have been used it with various
> >> languages. We might not be comfortable with Java 8 lambda (since
> transition
> >> to Java 8 is going slowly), but it's just a matter of familiarizing.
> >>
> >> Are there kind of best practices for Java 8 lambda? We can refer these
> to
> >> construct some guides / restrictions for Storm project.
> >>
> >> Thanks,
> >> Jungtaek Lim (HeartSaVioR)
> >>
> >> 2017년 1월 3일 (화) 오후 12:26, S G <sg.online.em...@gmail.com>님이 작성:
> >>
> >> > I have found several bad usages of Java 8 lambdas and many developers
> are
> >> > not comfortable using them.
> >> >
> >> > So we should use them only if it really makes the code beautiful and
> >> easier
> >> > to understand.
> >> >
> >> > My 2c,
> >> > Thanks
> >> > SG
> >> >
> >> >
> >> > On Mon, Dec 26, 2016 at 5:59 AM, Arun Mahadevan <ar...@apache.org>
> >> wrote:
> >> >
> >> > > The streams API implementation has limited usage of 1.8 features and
> >> can
> >> > > be easily ported to 1.7 if required. The examples are written in
> 1.8,
> >> the
> >> > > thought being users would stick to the Java 8 style usage (lambdas)
> >> from
> >> > > the beginning. If there is consensus we could also consider moving
> the
> >> > 1.x
> >> > > branch to JDK 8.
> >> > >
> >> > > Anyways would like interested folks to start reviewing the changes
> so
> >> > that
> >> > > we can take it forward.
> >> > >
> >> > > Thanks,
> >> > > Arun
> >> > >
> >> > >
> >> > > On 12/23/16, 10:09 AM, "Jungtaek Lim" <kabh...@gmail.com> wrote:
> >> > >
> >> > > >FYI, I've realized that internal of Stream API (pull request)
> relies
> >> on
> >> > > JDK
> >> > > >8 (what I've found is 'static method in interface' and maybe more)
> so
> >> > for
> >> > > >now Stream API is expected to be included for at least Storm 2.0.0
> >> > unless
> >> > > >the PR is modified to fit to JDK 7.
> >> > > >
> >> > > >- Jungtaek Lim (HeartSaVioR)
> >> > > >
> >> > > >2016년 12월 21일 (수) 오전 9:40, Jungtaek Lim <kabh...@gmail.com>님이 작성:
> >> > > >
> >> > > >> Thanks Manu and Taylor for giving your opinions.
> >> > > >>
> >> > > >> - Storm SQL improvement
> >> > > >>
> >> > > >> There're some huge PRs available but there're all about
> improvement
> >> > > which
> >> > > >> shouldn't be blocker for releasing 1.1.0. (I'd like to also
> include
> >> > > them to
> >> > > >> 1.1.0 but not sure it can be happen really soon.)
> >> > > >> I'll send a request for reviewing about pending Storm SQL PRs.
> >> > > >>
> >> > > >> Only one issue (STORM-2200) is linked to release 1.1.0 epic
> which is
> >> > > >> blocker for me.
> >> > > >>
> >> > > >> - Java port
> >> > > >>
> >> > > >> I also had some developers saying 'If core of Storm were written
> by
> >> > > Java,
> >> > > >> I could experiment and even contribute on something'. I was one
> of
> >> > them,
> >> > > >> and to be honest, I'm still a beginner of Clojure. Moving to
> Java 8
> >> > also
> >> > > >> gives great functionalities for us, so Java port is what I think
> the
> >> > > most
> >> > > >> important thing among the huge works now in progress. Ideally,
> and
> >> > > >> hopefully, I'd like to see us focus on this and make this happen
> at
> >> > the
> >> > > >> very early next year.
> >> > > >> (Yes we should do some manual tests and maybe some refactoring
> too.)
> >> > > >>
> >> > > >> - Metrics V2
> >> > > >>
> >> > > >> I'm not sure when we plan to release Storm 1.2.0, but given that
> >> > > there're
> >> > > >> only two things left (logviewer / ui) for completing port work
> >> (except
> >> > > >> tests) I guess Storm 2.0.0 might be happen earlier.
> >> > > >> Taylor, when do you expect metrics V2 will be available for
> >> reviewing?
> >> > > >>
> >> > > >> - Stream API
> >> > > >>
> >> > > >> With labeling as experiment or annotating with evolving, we could
> >> > > include
> >> > > >> the first version to next minor excluding 1.1.0. (We could even
> >> > include
> >> > > >> this to 1.1.0 if we start reviewing this very soon.)
> >> > > >>
> >> > > >> I'd like to hear others' opinions as well.
> >> > > >>
> >> > > >> Thanks,
> >> > > >> Jungtaek Lim (HeartSaVioR)
> >> > > >>
> >> > > >> 2016년 12월 21일 (수) 오전 7:33, P. Taylor Goetz <ptgo...@gmail.com>님이
> >> 작성:
> >> > > >>
> >> > > >> Hi Jungtaek,
> >> > > >>
> >> > > >> > - Beam runner
> >> > > >>
> >> > > >> There’s not been much activity around this, and I haven’t had
> much
> >> > time
> >> > > to
> >> > > >> work on it recently, but there’s a decent foundation to build
> upon.
> >> So
> >> > > it
> >> > > >> would be fairly easy for others to start contributing to that
> >> effort.
> >> > > >> There’s also interest from the Beam community in that runner, so
> one
> >> > > >> possibility is to move that effort to the Apache Beam project.
> >> > > >>
> >> > > >> This is very preliminary work, so I don’t have a good handle on
> what
> >> > the
> >> > > >> target release would be.
> >> > > >>
> >> > > >> > - Metrics renewal
> >> > > >>
> >> > > >>
> >> > > >> This is what I’ve been referring to as “metrics_v2”. This is
> >> > progressing
> >> > > >> fairly well with support for multiple reporters (e.g. Graphite,
> >> > Ganglia,
> >> > > >> console, etc.), worker metrics, disruptor metrics, etc.
> >> > > >>
> >> > > >> I would like to target this work for 1.2.0.
> >> > > >>
> >> > > >> > - Java port
> >> > > >>
> >> > > >> This effort seems to have picked up (for example Bobby’s
> conversion
> >> of
> >> > > >> Nimbus, etc.) and is progressing steadily. It’s taken a lot
> longer
> >> > than
> >> > > >> initially thought, but a lot of that can be attributed to the ebb
> >> and
> >> > > flow
> >> > > >> of people’s availability to do the work.
> >> > > >>
> >> > > >> > - Storm SQL improvement (Streaming SQL in future)
> >> > > >>
> >> > > >> You’ve been spearheading most of the work here, so I’d delegate
> to
> >> you
> >> > > for
> >> > > >> your opinion on where it stands. If you need additional reviews,
> >> just
> >> > > ask
> >> > > >> on list or via GitHub (e.g. “[REVIEW REQUEST]” in the subject
> line
> >> > might
> >> > > >> help get attention).
> >> > > >>
> >> > > >> My thinking has been that this could be included in the 1.1.0
> >> release.
> >> > > Is
> >> > > >> there a set of JIRA issues you would like to include in order to
> >> make
> >> > > that
> >> > > >> happen?
> >> > > >>
> >> > > >> > - Stream API
> >> > > >>
> >> > > >> This seems to have stalled a bit, though there seems to be a lot
> of
> >> > > >> interest around it. I think we all would agree that when
> >> introducing a
> >> > > new
> >> > > >> API for building topologies, it’s important that we get right
> from
> >> the
> >> > > >> start and have strong buy-in from the development community. I
> would
> >> > > >> encourage anyone interested in the Streams API to review the
> >> proposal
> >> > > and
> >> > > >> initial code.
> >> > > >>
> >> > > >> I think it is close, but I’m not sure what release to target.
> >> Possibly
> >> > > the
> >> > > >> 2.0 release?
> >> > > >>
> >> > > >> Re: 1.1.0 Release
> >> > > >>
> >> > > >> STORM-2176 is a fairly big concern of mine since the feature it
> >> > involves
> >> > > >> was introduced in 1.0.0 and did not work then nor in any
> subsequent
> >> or
> >> > > >> future releases (may not be a problem in 2.0). Unfortunately, as
> >> > you’ve
> >> > > >> seen, finding the root cause is elusive. That issue could
> definitely
> >> > use
> >> > > >> more eyes.
> >> > > >>
> >> > > >> -Taylor
> >> > > >>
> >> > > >>
> >> > > >> > On Dec 20, 2016, at 2:19 AM, Jungtaek Lim <kabh...@gmail.com>
> >> > wrote:
> >> > > >> >
> >> > > >> > Hi devs,
> >> > > >> >
> >> > > >> > I'm seeing lots of huge works in parallel, and we individual
> are
> >> > busy
> >> > > >> > regarding each work so common works (review, release,
> >> documentation,
> >> > > >> etc.)
> >> > > >> > have been not made in progress for several months.
> >> > > >> >
> >> > > >> > - Beam runner
> >> > > >> > - Metrics renewal
> >> > > >> > - Java port
> >> > > >> > - Storm SQL improvement (Streaming SQL in future)
> >> > > >> > - Stream API
> >> > > >> >
> >> > > >> > IMHO, it would be better to set target versions for them, and
> set
> >> a
> >> > > >> roadmap
> >> > > >> > (per version), and prioritize based on roadmap.
> >> > > >> >
> >> > > >> > Stream API (very first version), and Storm SQL improvement are
> >> > waiting
> >> > > >> for
> >> > > >> > review, and personally I would like to publish them soon.
> >> > > >> >
> >> > > >> > If we're OK to have 2.0.0 without adding much features, I'm in
> >> favor
> >> > > of
> >> > > >> > concentrating Java port work (postponing other things except
> >> > releasing
> >> > > >> 1.x
> >> > > >> > version line) and moving to Apache Storm 2.0.0 really soon.
> >> > > >> > (I'm even OK we decide to postpone some clojure files to be
> >> > addressed
> >> > > >> after
> >> > > >> > 2.0.0.)
> >> > > >> > Actually we're suffering other annoying issue: JDK 7 (1.x) vs 8
> >> > (2.x)
> >> > > >> which
> >> > > >> > is other reason to move to 2.x quickly.
> >> > > >> >
> >> > > >> > I'd be really happy if we have metrics renewal and beam runner,
> >> but
> >> > > I'm
> >> > > >> not
> >> > > >> > sure when they're available to be published. Do we have any
> >> updates
> >> > > here?
> >> > > >> >
> >> > > >> > What do you think? It might be ideal, and/or broader discussion
> >> but
> >> > we
> >> > > >> > haven't discussed our plan / vision for a long time so better
> to
> >> > give
> >> > > it
> >> > > >> a
> >> > > >> > try.
> >> > > >> >
> >> > > >> > Thanks,
> >> > > >> > Jungtaek Lim (HeartSaVioR)
> >> > > >>
> >> > > >>
> >> > >
> >> > >
> >> > >
> >> >
> >>
>
>
>
