Its more about getting used to and judiciously deciding when to use functional
v/s procedural approach. Badly designed APIs can get more complicated when
combined with lambdas, but overall what I observe is lambdas and functional
style can greatly improve the readability (with many other benefits) if used
properly.
A simple word count pipeline like below can get too verbose when expressed
without lambdas.
Stream<String> stream = …
stream
.flatMap(s -> Arrays.asList(s.split(" ")))
.mapToPair(w -> Pair.of(w, 1))
.countByKey()
.filter(x -> x.getSecond() >= 5)
.print();
*Without lambda*
stream
.flatMap(new FlatMapFunction<String, String>() {
@Override
public Iterable<String> apply(String s) {
return Arrays.asList(s.split(" "));
}
})
.mapToPair(new PairFunction<String, String, Integer>() {
@Override
public Pair<String, Integer> apply(String w) {
return Pair.of(w, 1);
}
})
.countByKey()
.filter(new Predicate<Pair<String, Long>>() {
@Override
public boolean test(Pair<String, Long> x) {
return x.getSecond() >= 5;
}
})
.print();
Thanks,
Arun
On 1/3/17, 10:21 AM, "S G" <sg.online.em...@gmail.com> wrote:
>One bad usage I have found is missing types for lambda arguments.
>When the types for arguments are not provided in the code itself, github or
>vi-editor browsing of such code becomes impossible.
>
>You must have an IDE like Eclipse to infer the types for such cases and
>that becomes a big limitation to browse such code. I have not looked into
>the storm code-base to find such occurrences but the above is just a
>general observation from the lambda code usages.
>
>
>Secondly, lambdas tend to generate bad stack-traces on exceptions.
>
>
>And lastly, functional programming is not natural to many Java developers.
>https://zeroturnaround.com/rebellabs/how-to-avoid-ruining-your-world-with-lambdas-in-java-8/
>hints at some of these problems. Java code is usually very easy to
>understand. But once you throw in more than a single level of lambdas,
>things tend to become more and more confusing for several developers. (This
>last point reflects the thoughts from some of my fellow developers, not
>representative of all the devs)
>
>
>Thanks
>SG
>
>
>On Mon, Jan 2, 2017 at 8:26 PM, Jungtaek Lim <kabh...@gmail.com> wrote:
>
>> Hi S G,
>>
>> I'd be happy if you could elaborate your opinion. Did you found bad usages
>> from master branch of Storm code?
>>
>> Regarding comfortable of lambda, IMHO, I don't think many users are
>> unfamiliar with lambda, since they should have been used it with various
>> languages. We might not be comfortable with Java 8 lambda (since transition
>> to Java 8 is going slowly), but it's just a matter of familiarizing.
>>
>> Are there kind of best practices for Java 8 lambda? We can refer these to
>> construct some guides / restrictions for Storm project.
>>
>> Thanks,
>> Jungtaek Lim (HeartSaVioR)
>>
>> 2017년 1월 3일 (화) 오후 12:26, S G <sg.online.em...@gmail.com>님이 작성:
>>
>> > I have found several bad usages of Java 8 lambdas and many developers are
>> > not comfortable using them.
>> >
>> > So we should use them only if it really makes the code beautiful and
>> easier
>> > to understand.
>> >
>> > My 2c,
>> > Thanks
>> > SG
>> >
>> >
>> > On Mon, Dec 26, 2016 at 5:59 AM, Arun Mahadevan <ar...@apache.org>
>> wrote:
>> >
>> > > The streams API implementation has limited usage of 1.8 features and
>> can
>> > > be easily ported to 1.7 if required. The examples are written in 1.8,
>> the
>> > > thought being users would stick to the Java 8 style usage (lambdas)
>> from
>> > > the beginning. If there is consensus we could also consider moving the
>> > 1.x
>> > > branch to JDK 8.
>> > >
>> > > Anyways would like interested folks to start reviewing the changes so
>> > that
>> > > we can take it forward.
>> > >
>> > > Thanks,
>> > > Arun
>> > >
>> > >
>> > > On 12/23/16, 10:09 AM, "Jungtaek Lim" <kabh...@gmail.com> wrote:
>> > >
>> > > >FYI, I've realized that internal of Stream API (pull request) relies
>> on
>> > > JDK
>> > > >8 (what I've found is 'static method in interface' and maybe more) so
>> > for
>> > > >now Stream API is expected to be included for at least Storm 2.0.0
>> > unless
>> > > >the PR is modified to fit to JDK 7.
>> > > >
>> > > >- Jungtaek Lim (HeartSaVioR)
>> > > >
>> > > >2016년 12월 21일 (수) 오전 9:40, Jungtaek Lim <kabh...@gmail.com>님이 작성:
>> > > >
>> > > >> Thanks Manu and Taylor for giving your opinions.
>> > > >>
>> > > >> - Storm SQL improvement
>> > > >>
>> > > >> There're some huge PRs available but there're all about improvement
>> > > which
>> > > >> shouldn't be blocker for releasing 1.1.0. (I'd like to also include
>> > > them to
>> > > >> 1.1.0 but not sure it can be happen really soon.)
>> > > >> I'll send a request for reviewing about pending Storm SQL PRs.
>> > > >>
>> > > >> Only one issue (STORM-2200) is linked to release 1.1.0 epic which is
>> > > >> blocker for me.
>> > > >>
>> > > >> - Java port
>> > > >>
>> > > >> I also had some developers saying 'If core of Storm were written by
>> > > Java,
>> > > >> I could experiment and even contribute on something'. I was one of
>> > them,
>> > > >> and to be honest, I'm still a beginner of Clojure. Moving to Java 8
>> > also
>> > > >> gives great functionalities for us, so Java port is what I think the
>> > > most
>> > > >> important thing among the huge works now in progress. Ideally, and
>> > > >> hopefully, I'd like to see us focus on this and make this happen at
>> > the
>> > > >> very early next year.
>> > > >> (Yes we should do some manual tests and maybe some refactoring too.)
>> > > >>
>> > > >> - Metrics V2
>> > > >>
>> > > >> I'm not sure when we plan to release Storm 1.2.0, but given that
>> > > there're
>> > > >> only two things left (logviewer / ui) for completing port work
>> (except
>> > > >> tests) I guess Storm 2.0.0 might be happen earlier.
>> > > >> Taylor, when do you expect metrics V2 will be available for
>> reviewing?
>> > > >>
>> > > >> - Stream API
>> > > >>
>> > > >> With labeling as experiment or annotating with evolving, we could
>> > > include
>> > > >> the first version to next minor excluding 1.1.0. (We could even
>> > include
>> > > >> this to 1.1.0 if we start reviewing this very soon.)
>> > > >>
>> > > >> I'd like to hear others' opinions as well.
>> > > >>
>> > > >> Thanks,
>> > > >> Jungtaek Lim (HeartSaVioR)
>> > > >>
>> > > >> 2016년 12월 21일 (수) 오전 7:33, P. Taylor Goetz <ptgo...@gmail.com>님이
>> 작성:
>> > > >>
>> > > >> Hi Jungtaek,
>> > > >>
>> > > >> > - Beam runner
>> > > >>
>> > > >> There’s not been much activity around this, and I haven’t had much
>> > time
>> > > to
>> > > >> work on it recently, but there’s a decent foundation to build upon.
>> So
>> > > it
>> > > >> would be fairly easy for others to start contributing to that
>> effort.
>> > > >> There’s also interest from the Beam community in that runner, so one
>> > > >> possibility is to move that effort to the Apache Beam project.
>> > > >>
>> > > >> This is very preliminary work, so I don’t have a good handle on what
>> > the
>> > > >> target release would be.
>> > > >>
>> > > >> > - Metrics renewal
>> > > >>
>> > > >>
>> > > >> This is what I’ve been referring to as “metrics_v2”. This is
>> > progressing
>> > > >> fairly well with support for multiple reporters (e.g. Graphite,
>> > Ganglia,
>> > > >> console, etc.), worker metrics, disruptor metrics, etc.
>> > > >>
>> > > >> I would like to target this work for 1.2.0.
>> > > >>
>> > > >> > - Java port
>> > > >>
>> > > >> This effort seems to have picked up (for example Bobby’s conversion
>> of
>> > > >> Nimbus, etc.) and is progressing steadily. It’s taken a lot longer
>> > than
>> > > >> initially thought, but a lot of that can be attributed to the ebb
>> and
>> > > flow
>> > > >> of people’s availability to do the work.
>> > > >>
>> > > >> > - Storm SQL improvement (Streaming SQL in future)
>> > > >>
>> > > >> You’ve been spearheading most of the work here, so I’d delegate to
>> you
>> > > for
>> > > >> your opinion on where it stands. If you need additional reviews,
>> just
>> > > ask
>> > > >> on list or via GitHub (e.g. “[REVIEW REQUEST]” in the subject line
>> > might
>> > > >> help get attention).
>> > > >>
>> > > >> My thinking has been that this could be included in the 1.1.0
>> release.
>> > > Is
>> > > >> there a set of JIRA issues you would like to include in order to
>> make
>> > > that
>> > > >> happen?
>> > > >>
>> > > >> > - Stream API
>> > > >>
>> > > >> This seems to have stalled a bit, though there seems to be a lot of
>> > > >> interest around it. I think we all would agree that when
>> introducing a
>> > > new
>> > > >> API for building topologies, it’s important that we get right from
>> the
>> > > >> start and have strong buy-in from the development community. I would
>> > > >> encourage anyone interested in the Streams API to review the
>> proposal
>> > > and
>> > > >> initial code.
>> > > >>
>> > > >> I think it is close, but I’m not sure what release to target.
>> Possibly
>> > > the
>> > > >> 2.0 release?
>> > > >>
>> > > >> Re: 1.1.0 Release
>> > > >>
>> > > >> STORM-2176 is a fairly big concern of mine since the feature it
>> > involves
>> > > >> was introduced in 1.0.0 and did not work then nor in any subsequent
>> or
>> > > >> future releases (may not be a problem in 2.0). Unfortunately, as
>> > you’ve
>> > > >> seen, finding the root cause is elusive. That issue could definitely
>> > use
>> > > >> more eyes.
>> > > >>
>> > > >> -Taylor
>> > > >>
>> > > >>
>> > > >> > On Dec 20, 2016, at 2:19 AM, Jungtaek Lim <kabh...@gmail.com>
>> > wrote:
>> > > >> >
>> > > >> > Hi devs,
>> > > >> >
>> > > >> > I'm seeing lots of huge works in parallel, and we individual are
>> > busy
>> > > >> > regarding each work so common works (review, release,
>> documentation,
>> > > >> etc.)
>> > > >> > have been not made in progress for several months.
>> > > >> >
>> > > >> > - Beam runner
>> > > >> > - Metrics renewal
>> > > >> > - Java port
>> > > >> > - Storm SQL improvement (Streaming SQL in future)
>> > > >> > - Stream API
>> > > >> >
>> > > >> > IMHO, it would be better to set target versions for them, and set
>> a
>> > > >> roadmap
>> > > >> > (per version), and prioritize based on roadmap.
>> > > >> >
>> > > >> > Stream API (very first version), and Storm SQL improvement are
>> > waiting
>> > > >> for
>> > > >> > review, and personally I would like to publish them soon.
>> > > >> >
>> > > >> > If we're OK to have 2.0.0 without adding much features, I'm in
>> favor
>> > > of
>> > > >> > concentrating Java port work (postponing other things except
>> > releasing
>> > > >> 1.x
>> > > >> > version line) and moving to Apache Storm 2.0.0 really soon.
>> > > >> > (I'm even OK we decide to postpone some clojure files to be
>> > addressed
>> > > >> after
>> > > >> > 2.0.0.)
>> > > >> > Actually we're suffering other annoying issue: JDK 7 (1.x) vs 8
>> > (2.x)
>> > > >> which
>> > > >> > is other reason to move to 2.x quickly.
>> > > >> >
>> > > >> > I'd be really happy if we have metrics renewal and beam runner,
>> but
>> > > I'm
>> > > >> not
>> > > >> > sure when they're available to be published. Do we have any
>> updates
>> > > here?
>> > > >> >
>> > > >> > What do you think? It might be ideal, and/or broader discussion
>> but
>> > we
>> > > >> > haven't discussed our plan / vision for a long time so better to
>> > give
>> > > it
>> > > >> a
>> > > >> > try.
>> > > >> >
>> > > >> > Thanks,
>> > > >> > Jungtaek Lim (HeartSaVioR)
>> > > >>
>> > > >>
>> > >
>> > >
>> > >
>> >
>>
