Not the DataSet but the JoinDataSet and the CoGroupDataSet do in the form
of an apply function.
On Tue, Feb 9, 2016 at 11:09 AM, Stefano Baghino <
stefano.bagh...@radicalbit.io> wrote:
> Sure, it was just a draft. I agree that filter and mapPartition make sense,
> but coGroup and join don't
This looks like a good design to me :-) The only thing is that it is not
complete. For example, the filter, mapPartition, coGroup and join functions
are missing.
Cheers,
Till
On Tue, Feb 9, 2016 at 1:18 AM, Stefano Baghino <
stefano.bagh...@radicalbit.io> wrote:
> What do you think of
Sure, it was just a draft. I agree that filter and mapPartition make sense,
but coGroup and join don't look like they take a function.
On Tue, Feb 9, 2016 at 10:08 AM, Till Rohrmann wrote:
> This looks like a good design to me :-) The only thing is that it is not
>
I see, thanks for the tip! I'll work on it; meanwhile, I've added some
functions and Scaladoc:
https://github.com/radicalbit/flink/blob/1159-implicit/flink-scala/src/main/scala/org/apache/flink/api/scala/extensions/package.scala
On Tue, Feb 9, 2016 at 12:01 PM, Till Rohrmann
I agree with you, but I acknowledge that there may be concerns regarding
the stability of the API. Perhaps the rationale behind the proposal of
Stephan and Till is to provide it as an extension to test how the
developers feel about it. It would be ideal to have a larger feedback from
the
Thanks for bringing this up Stefano, it would a very welcome addition
indeed.
I like the approach of having extensions through implicits as well. IMHO
though this should be the default
behavior, without the need to add another import.
On Tue, Feb 9, 2016 at 1:29 PM, Stefano Baghino <
Hi Till,
I do agree with your point, so much so so that at the time being I'd
suggest to keeping these additions as optional, up to the end-user to
opt-in.
Adding them by default would effectively be an addition to the DataSet API
(despite being separated at a source file level).
I think your
I think that if we make the class part of the default ".api.scala" package
object, we effectively add these methods to DataSet.scala, because they
will be always be available on the data set.
If we want to retain the liberty to not commit to this change now, then we
should probably ask users to
What we could do is to add the implicit class to the package object of
org.apache.flink.api.scala. Since one always has to import this package in
order to have the proper TypeInformations, you wouldn’t have to import the
extension explicitly.
The thing with the API is that you don’t want to break
I like the idea to support partial functions with Flink’s Scala API.
However, I think that breaking the API and making it inconsistent with
respect to the Java API is not the best option. I would rather be in favour
of the first proposal where we add a new method xxxWith via implicit
conversions.
It took me a little time but I was able to put together some code.
In this commit I just added a few methods renamed to prevent overloading,
thus usable with PartialFunction instead of functions:
https://github.com/radicalbit/flink/commit/aacd59e0ce98cccb66d48a30d07990ac8f345748
In this other
Hi Stephan,
thank you for the quick reply and for your feedback; I agree with you that
breaking changes have to taken very seriously.
The rationale behind my proposal is that Scala users are already accustomed
to higher-order functions that manipulate collections and it would
beneficial for them
Hello everybody,
as I'm getting familiar with Flink I've found a possible improvement to the
Scala APIs: in Scala it's a common pattern to perform tuple extraction
using pattern matching, making functions working on tuples more readable,
like this:
// referring to the mail count example in the
Hi!
Would be nice to support that, agreed.
Such a fundamental break in the API worries me a bit, though - I would opt
for a non-breaking addition.
Wrapping the RichFunctions into Scala functions (which are actually wrapped
as rich functions) with implicits seems like a workaround for something
14 matches
Mail list logo