Re: Case style anonymous functions not supported by Scala API

Not the DataSet but the JoinDataSet and the CoGroupDataSet do in the form of an apply function. ​ On Tue, Feb 9, 2016 at 11:09 AM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > Sure, it was just a draft. I agree that filter and mapPartition make sense, > but coGroup and join don't

Re: Case style anonymous functions not supported by Scala API

This looks like a good design to me :-) The only thing is that it is not complete. For example, the filter, mapPartition, coGroup and join functions are missing. Cheers, Till ​ On Tue, Feb 9, 2016 at 1:18 AM, Stefano Baghino < stefano.bagh...@radicalbit.io> wrote: > What do you think of

Re: Case style anonymous functions not supported by Scala API

Sure, it was just a draft. I agree that filter and mapPartition make sense, but coGroup and join don't look like they take a function. On Tue, Feb 9, 2016 at 10:08 AM, Till Rohrmann wrote: > This looks like a good design to me :-) The only thing is that it is not >

Re: Case style anonymous functions not supported by Scala API

I see, thanks for the tip! I'll work on it; meanwhile, I've added some functions and Scaladoc: https://github.com/radicalbit/flink/blob/1159-implicit/flink-scala/src/main/scala/org/apache/flink/api/scala/extensions/package.scala On Tue, Feb 9, 2016 at 12:01 PM, Till Rohrmann

Re: Case style anonymous functions not supported by Scala API

I agree with you, but I acknowledge that there may be concerns regarding the stability of the API. Perhaps the rationale behind the proposal of Stephan and Till is to provide it as an extension to test how the developers feel about it. It would be ideal to have a larger feedback from the

Re: Case style anonymous functions not supported by Scala API

Thanks for bringing this up Stefano, it would a very welcome addition indeed. I like the approach of having extensions through implicits as well. IMHO though this should be the default behavior, without the need to add another import. On Tue, Feb 9, 2016 at 1:29 PM, Stefano Baghino <

Re: Case style anonymous functions not supported by Scala API

Hi Till, I do agree with your point, so much so so that at the time being I'd suggest to keeping these additions as optional, up to the end-user to opt-in. Adding them by default would effectively be an addition to the DataSet API (despite being separated at a source file level). I think your

Re: Case style anonymous functions not supported by Scala API

I think that if we make the class part of the default ".api.scala" package object, we effectively add these methods to DataSet.scala, because they will be always be available on the data set. If we want to retain the liberty to not commit to this change now, then we should probably ask users to

Re: Case style anonymous functions not supported by Scala API

What we could do is to add the implicit class to the package object of org.apache.flink.api.scala. Since one always has to import this package in order to have the proper TypeInformations, you wouldn’t have to import the extension explicitly. The thing with the API is that you don’t want to break

Re: Case style anonymous functions not supported by Scala API

I like the idea to support partial functions with Flink’s Scala API. However, I think that breaking the API and making it inconsistent with respect to the Java API is not the best option. I would rather be in favour of the first proposal where we add a new method xxxWith via implicit conversions.

Re: Case style anonymous functions not supported by Scala API

It took me a little time but I was able to put together some code. In this commit I just added a few methods renamed to prevent overloading, thus usable with PartialFunction instead of functions: https://github.com/radicalbit/flink/commit/aacd59e0ce98cccb66d48a30d07990ac8f345748 In this other

Re: Case style anonymous functions not supported by Scala API

Hi Stephan, thank you for the quick reply and for your feedback; I agree with you that breaking changes have to taken very seriously. The rationale behind my proposal is that Scala users are already accustomed to higher-order functions that manipulate collections and it would beneficial for them

Case style anonymous functions not supported by Scala API

Hello everybody, as I'm getting familiar with Flink I've found a possible improvement to the Scala APIs: in Scala it's a common pattern to perform tuple extraction using pattern matching, making functions working on tuples more readable, like this: // referring to the mail count example in the

Re: Case style anonymous functions not supported by Scala API

Hi! Would be nice to support that, agreed. Such a fundamental break in the API worries me a bit, though - I would opt for a non-breaking addition. Wrapping the RichFunctions into Scala functions (which are actually wrapped as rich functions) with implicits seems like a workaround for something