Re: [DISCUSS] FLIP-39: Flink ML pipeline and ML libs

Hi, Some portion of the code could be migrated to the new Table API no? I am saying that because the new API design is based on scikit-learn and the old one was also inspired by it. Best, Stavros On Wed, May 22, 2019 at 1:24 PM Shaoxuan Wang wrote: > Another consensus (from the offline discussi

Re: [DISCUSS] FLIP-23 Model Serving

; >>>> flexibility and gives user the freedom to create his own types without > >>>> breaking underlying framework. > >>>> > >>>> Ad 4) @Boris: I made this point not about the serialization format but > >>>> how the library w

Re: [DISCUSS] FLIP-23 Model Serving

y) follow this > approach. > > 5) I'm skeptical about using queryable state to expose metrics. Did you > consider using Flink's metrics system [1]? It is easily configurable and we > provided several reporters that export the metrics. > > What do you think? > Best, F

[DISCUSS] FLIP-23 Model Serving

Hi guys, Let's discuss the new FLIP proposal for model serving over Flink. The idea is to combine previous efforts there and provide a library on top of Flink for serving models. https://cwiki.apache.org/confluence/display/FLINK/FLIP-23+-+Model+Serving Code from previous efforts can be found her

Re: [ANNOUNCE] New Flink PMC member: Tzu-Li (Gordon) Tai

Congrats Gordon! On Wed, Jul 12, 2017 at 5:29 AM, Evans Ye wrote: > Congrats Gordon! > > Tzu-Li (Gordon) Tai 於 2017年7月12日 週三，上午2:28寫道： > > > Thanks everyone :) > > It has always been great working with you all! > > Looking forward to all the future endeavors to be done in continuing to > > push

Re: [DISCUSS] FLIP proposal for Model Serving over Flink

is > > > > year? > > > > > > > > On Fri, Jun 30, 2017 at 6:04 PM, Fabian Hueske > > > wrote: > > > >> > > > >> Yes, I know that Theo is engaged in the ML efforts but wasn't sure > how > > > >> much > &

Re: [DISCUSS] FLIP proposal for Model Serving over Flink

people are contributing to the model serving module, the number of > committers should hopefully grow after some time. > > Best, Fabian > > 2017-06-30 10:58 GMT+02:00 Stavros Kontopoulos : > > > Hi all, > > > > After coordinating with Theodore Vasiloudis and the gu

[DISCUSS] FLIP proposal for Model Serving over Flink

Hi all, After coordinating with Theodore Vasiloudis and the guys behind the Flink Model Serving effort (Eron, Radicalbit people, Boris, Bas (ING)), we propose to start working on the model serving over Flink in a more official way. That translates to capturing design details in a FLIP document. P

Re: Re: Switch to Scala 2.11 as a default build profile

+10 I think it makes sense spark is also using 2.11 as the default for quite some time. On Thu, Jun 29, 2017 at 8:14 PM, Bowen Li wrote: > EMR's builtin Flink is always 1 or 2 versions behind Flink latest release. > We choose to install Flink on EMR ourselves. > > On Thu, Jun 29, 2017 at 2:37 AM

Re: FlinkML on slack

created an app to automate the invite process, now you can just use > > the following link > > to get an invite to the FlinkML Slack group: > > > > https://flinkml-invites.herokuapp.com/ > > > > Regards, > > Theodore > > > > On Tue, Jun 20, 2017

Re: [ANNOUNCE] New committer: Dawid Wysakowicz

Congratulations Dawid! On Tue, Jun 20, 2017 at 12:06 PM, Vasudevan, Ramkrishna S < ramkrishna.s.vasude...@intel.com> wrote: > Congratulations !! > > -Original Message- > From: Henry Saputra [mailto:henry.sapu...@gmail.com] > Sent: Tuesday, June 20, 2017 2:19 PM > To: dev@flink.apache.org

Re: FlinkML on slack

n I get an invitation for the slack channel. > > > > > > Thanks, > > > Shaoxuan > > > > > > > > > On Thu, Jun 8, 2017 at 3:56 AM, Stavros Kontopoulos < > > > st.kontopou...@gmail.com> wrote: > > > > > > > Hi al

Re: FlinkML on slack

mail.com> > wrote: > > > Hi Stravros, > > > > Could you also please add me to the Slack channel? My email id is: > > lokesh.amarn...@gmail.com. > > > > Thanks, > > Lokesh > > > > > > > > On Thu, Jun 15, 2017 at 6:27 PM,

Re: FlinkML on slack

Ziyad added. Stavros On Sun, Jun 11, 2017 at 4:45 PM, Ziyad Muhammed wrote: > Hi Stavros > > Could you please send me an invite to the slack channel? > > Best > Ziyad > > > On Sun, Jun 11, 2017 at 1:53 AM, Stavros Kontopoulos < > st.kontopou...@gmail.com&

Re: FlinkML on slack

s, > > Henry > > > On Thu, Jun 8, 2017 at 2:21 AM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > > > Hi Aljoscha, > > > > Slack is invite only to the best of my knowledge, I just sent you an > > invitation. > > > > Best,

Re: FlinkML on slack

@Ted Yu sure. On Thu, Jun 8, 2017 at 5:18 PM, Ted Yu wrote: > Hi Stavros, > Can you add me as well ? > > Thanks > > On Wed, Jun 7, 2017 at 12:56 PM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > > > Hi all, > > > > We took the init

Re: FlinkML on slack

invitations). Stavros On Thu, Jun 8, 2017 at 12:36 PM, Till Rohrmann wrote: > HI Stavros, > > could you also send me an invite. Thanks. > > Cheers, > Till > > On Thu, Jun 8, 2017 at 11:28 AM, Aljoscha Krettek > wrote: > > > Thanks! > > > >

Re: FlinkML on slack

t; On 7. Jun 2017, at 21:56, Stavros Kontopoulos > wrote: > > > > Hi all, > > > > We took the initiative to create the organization for FlinkML on slack > > (thnx Eron). > > There is now a channel for model-serving > > <https://docs.google.com/docume

FlinkML on slack

Hi all, We took the initiative to create the organization for FlinkML on slack (thnx Eron). There is now a channel for model-serving . Another is coming for flink-jpmml. You are invited to join the channels and

Re: interested in volunteering at Flink ML/Stream ML Project

Hi, If you are aware of the following you can skip it... There was a previous discussion about Flink ML recently and several people are working on it on different directions. Discussion: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Machine-Learning-on-Flink-Next-steps-td16334i

[jira] [Created] (FLINK-6668) Add flink history server to DCOS

Stavros Kontopoulos created FLINK-6668: -- Summary: Add flink history server to DCOS Key: FLINK-6668 URL: https://issues.apache.org/jira/browse/FLINK-6668 Project: Flink Issue Type: New

Re: Machine Learning on Flink - Next steps

gle doc, that is > >>> representing > >>> the selected directions and persons, who would like to drive or > >>> participate > >>> in the particular topic, in order to make this process transparent for > >>> community and sum up

Re: Machine Learning on Flink - Next steps

who expressed interest in the project. > > Would you be willing to lead that effort for the model serving project? > > Regards, > Theodore > > -- > Sent from a mobile device. May contain autocorrect errors. > > On Mar 19, 2017 3:49 AM, "Stavros Kontopoulos

Re: Machine Learning on Flink - Next steps

Hi all... I agree about the tensorflow integration it seems to be important from what I hear. Should we sign up somewhere for the working groups (gdcos)? I would like to start helping with the model serving feature. Best Regards, Stavros On Fri, Mar 17, 2017 at 10:34 PM, Gábor Hermann wrote: >

Re: Machine Learning on Flink - Next steps

Thanks Theodore, I'd vote for - Offline learning with Streaming API - Low-latency prediction serving Some comments... Online learning Good to have but my feeling is that it is not a strong requirement (if a requirement at all) across the industry right now. May become hot in the future. Offl

Re: [DISCUSS] Flink ML roadmap

ch of the pros/cons have already been discussed here, so I'll try > to > > >>> put there all the arguments mentioned in this thread. Feel free to > put > > >>> there more :) > > >>> > > >>> @Stavros: I agree we should take action fast.

Re: [DISCUSS] Code style / checkstyle

+1 to provide and enforcing a unified code style for both java and scala. Unification should apply when it makes sense like comments though. Eventually code base should be re-factored. I would vote for the one at a time module fix apporoach. Style guide should be part of any PR review. We could a

Re: [DISCUSS] Flink ML roadmap

gt;> During analysis something will finally arise. >> May be we can ask partners of Flink for cases? Data Artisans got results >> of customers survey: [1], ML better support is wanted, so we could ask >> what >> exactly is necessary. >> >> [1] http://data-artisa

Re: [DISCUSS] Flink ML roadmap

+100 for a design doc. Could we also set a roadmap after some time-boxed investigation captured in that document? We need action. Looking forward to work on this (whatever that might be) ;) Also are there any data supporting one direction or the other from a customer perspective? It would help to

Re: [DISCUSS] Flink ML roadmap

Ok I see. Suppose we solve all the critical issues. And suppose we dont go with the pure online model (although online ML has a potential)... should we move on with the current ML implementation which is for batch processing (to the best of my knowledge)? The parameter server problem is a long stan

Re: [DISCUSS] Flink ML roadmap

to that. > > > > > > > > In terms of features (a, d), I think we should first see the bigger > > > > picture. That is, it would be nice to discuss a clearer direction for > > > Flink > > > > ML. I've seen a lot of interest in contributing

[DISCUSS] Flink ML roadmap

(Resending with the appropriate topic) Hi, I would like to start a discussion about next steps for Flink ML. Currently there is a lot of work going on but needs a push forward. Some topics to discuss: a) How several features should be planned and get aligned with Flink releases. b) Priorities o

Flink ML

Hi, I would like to start a discussion about next steps for Flink ML. Currently there is a lot of work going on but needs a push forward. Some topics to discuss: a) How several features should be planned and get aligned with Flink releases. b) Priorities of what should be done. c) Basic guidelin

[jira] [Created] (FLINK-5841) Algorithms for each pipeline stage should handle NaN, infinity like in scikit-learn

Stavros Kontopoulos created FLINK-5841: -- Summary: Algorithms for each pipeline stage should handle NaN, infinity like in scikit-learn Key: FLINK-5841 URL: https://issues.apache.org/jira/browse/FLINK-5841

[jira] [Created] (FLINK-5785) Add an Imputer for preparing data

Stavros Kontopoulos created FLINK-5785: -- Summary: Add an Imputer for preparing data Key: FLINK-5785 URL: https://issues.apache.org/jira/browse/FLINK-5785 Project: Flink Issue Type: New

Re: Flink ML - NaN Handling

data I get a result like: DenseVector(0.34528405956977387, 0.5, NaN) ... which is reasonable given the implementation but should be allowed? On Sun, Feb 12, 2017 at 9:03 PM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Ok cool thnx Till. > > On Sun, Feb 12, 2017 at 4:5

Re: flink-ml test

Ok I missed the IT part, if it is an integration test class you could use mvn integration-test -DwildcardSuites= the mvn target is different. On Sat, Feb 11, 2017 at 12:20 AM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Weird for me it works. I can post the output he

Re: Flink ML - NaN Handling

> On Fri, Feb 10, 2017 at 11:48 PM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > > > Hello guys, > > > > Is there a story for this (might have been discussed earlier)? I see > > differences between scikit-learn and numpy. Do we standardize on > >

Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

Congrats! On Fri, Feb 10, 2017 at 9:11 PM, Matthias J. Sax wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Congrats! > > On 2/10/17 2:00 AM, Ufuk Celebi wrote: > > Hey everyone, > > > > I'm very happy to announce that the Flink PMC has accepted Stefan > > Richter to become a commi

Flink ML - NaN Handling

Hello guys, Is there a story for this (might have been discussed earlier)? I see differences between scikit-learn and numpy. Do we standardize on scikit-learn? PS. I am working on the preprocessing stuff. Best, Stavros

Re: flink-ml test

nk.ml On Fri, Jan 27, 2017 at 4:07 PM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > typo:remove the second BreezeMathSuite.. > > > On Fri, Jan 27, 2017 at 4:06 PM, Stavros Kontopoulos < > st.kontopou...@gmail.com> wrote: > >> Hi, >> >&g

Re: flink-ml test

Hi, For running a specific test under flink-ml: mvn test -DwildcardSuites=org.apache.flink.ml.math.BreezeMathSuite BreezeMathSuite Cheers, Stavros On Fri, Jan 27, 2017 at 12:01 PM, Driesprong, Fokko wrote: > Hi Anton, > > I'm curious what tests fail. I run the tests by using `mvn verify` in th

Re: flink-ml test

typo:remove the second BreezeMathSuite.. On Fri, Jan 27, 2017 at 4:06 PM, Stavros Kontopoulos < st.kontopou...@gmail.com> wrote: > Hi, > > For running a specific test under flink-ml: > mvn test -DwildcardSuites=org.apache.flink.ml.math.BreezeMathSuite > BreezeMathSuite

[jira] [Created] (FLINK-5588) Add a unit scaler based on different norms

Stavros Kontopoulos created FLINK-5588: -- Summary: Add a unit scaler based on different norms Key: FLINK-5588 URL: https://issues.apache.org/jira/browse/FLINK-5588 Project: Flink Issue

[jira] [Created] (FLINK-5525) Streaming Version of a Linear Regression model

Stavros Kontopoulos created FLINK-5525: -- Summary: Streaming Version of a Linear Regression model Key: FLINK-5525 URL: https://issues.apache.org/jira/browse/FLINK-5525 Project: Flink

Re: buffering in operators, implementing statistics

ng to write a program once and then > be > > able to run it on different runners. This brings more flexibility for > > users. It's not clear how this will play out in the long run but it's > very > > interesting to keep an eye on. > > > > For most o

Re: buffering in operators, implementing statistics

> maybe our energy is better spent on producing examples with real-world > applicability. I'm not against having an example for a count-min sketch, > I'm just worried that you might put your energy into something that is not > useful to a lot of people. > > Cheer

Re: buffering in operators, implementing statistics

2147. I already > commented on https://issues.apache.org/jira/browse/FLINK-2144, saying a > similar thing. > > What I would welcome very much is to add some well documented examples to > Flink that showcase how some of these operations can be written. > > Cheers, > Al

buffering in operators, implementing statistics

Hi guys, I would like to push forward the work here: https://issues.apache.org/jira/browse/FLINK-2147 Can anyone more familiar with streaming api verify if this could be a mature task. The intention is to summarize data over a window like in the case of StreamGroupedFold. Specifically implement c

withBroadcastSet for a DataStream missing?

H i am new here... I am trying to implement online k-means as here https://databricks.com/blog/2015/01/28/introducing-streaming-k-means-in-spark-1-2.html with flink. I dont see anywhere a withBroadcastSet call to save intermediate results is this currently supported? Is intermediate results state