Re: [VOTE] Flink Project Bylaws

2019-08-23 Thread Kostas Tzoumas
+1 On Thu, Aug 22, 2019 at 5:29 PM jincheng sun wrote: > +1 > > Becket Qin 于2019年8月22日 周四16:22写道: > > > Hi All, so far the votes count as following: > > > > +1 (Binding): 13 (Aljoscha, Fabian, Kurt, Till, Timo, Max, Stephan, > > Gordon, Robert, Ufuk, Chesnay, Shaoxuan, Henry) > > +0 (Binding): 1

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-25 Thread Kostas Tzoumas
+1 great idea! On Fri, Jan 25, 2019 at 11:28 AM Till Rohrmann wrote: > +1, I like this idea very much. > > On Fri, Jan 25, 2019 at 5:38 AM Jark Wu wrote: > > > a big +1 to this! > > > > An user-zh mailing list will help more Chinese speaking users and there > > will be more people willing to h

Re: [DISCUSS] Proposed updates to Flink project site

2017-01-10 Thread Kostas Tzoumas
Love it! I would merge this asap Some comments (not blockers) - I was looking for the downloads page for a minute, until I saw that it is linked via the big blue button :) Did anyone have the same problem? - Do we still link to the wiki? - Maybe too much info is buried under the Community & Proje

Re: [DISCUSS] ElasticSearch in Flink Strategy

2017-01-09 Thread Kostas Tzoumas
+1 for keeping the ES connector in Flink On Mon, Jan 9, 2017 at 2:53 PM, Robert Metzger wrote: > Thanks for bringing this up again Flavio. > > The good thing about a common base for all ES connectors is that we can fix > the issue for all ES versions (ideally). > > Thanks a lot for taking care o

Who's hiring, December 2016

2016-12-16 Thread Kostas Tzoumas
Hi folks, As promised, here is the first thread for Flink-related job positions. If your organization is hiring people on Flink-related positions do reply to this thread with a link for applications. data Artisans is hiring on multiple technical positions. Help us build Flink, and help our custom

[ANNOUNCE] New Flink community mailing list

2016-12-14 Thread Kostas Tzoumas
Hi everyone, We have created a new Flink mailing lists, commun...@flink.apache.org where we can post everything related to the broader Flink community including job offers, upcoming meetups and conferences, exciting reads, and everything else that is deemed worthy for the greater Flink community.

Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

2016-12-13 Thread Kostas Tzoumas
It seems that several folks are excited about the idea - but there is still a concern on whether this would be spam for the dev@ and user@ lists (which I share) As a compromise, I propose to request a new mailing list ( commun...@flink.apache.org) which we can use for this purpose, and also to pos

Re: [DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

2016-12-06 Thread Kostas Tzoumas
yes, of course! On Tue, Dec 6, 2016 at 12:54 PM, Márton Balassi wrote: > +1. It keeps it both organized and to a reasonable minimum overhead. > > Would you volunteer for starting the mail thread each month then, Kostas? > > Best, > > Marton > > On Tue, Dec 6, 2016

[DISCUSS] "Who's hiring on Flink" monthly thread in the mailing lists?

2016-12-06 Thread Kostas Tzoumas
Hi folks, I'd like to see how the community feels about a monthly "Who is hiring on Flink" email thread on the dev@ and user@ mailing lists where folks can post job positions related to Flink. I personally think that posting individual job offerings in the mailing list is off-topic (hence I have

Flink survey by data Artisans

2016-11-18 Thread Kostas Tzoumas
Hi everyone! The Apache Flink community has evolved quickly over the past 2+ years, and there are now many production Flink deployments in organizations of all sizes. This is both exciting and humbling :-) data Artisans is running a brief survey to understand Apache Flink usage and the needs of

[ANNOUNCE] Flink Forward 2016: First round of speakers and sessions is out

2016-07-25 Thread Kostas Tzoumas
Hi everyone, I wanted to share this with the community: we have announced the first round of speakers and sessions of Flink Forward 2016, and it looks amazing! Check it out here: http://flink-forward.org/program/sessions/ This year we have a great mix of use case talks (e.g., by Netflix, Alibaba

Re: [DISCUSS] Putting Flink user names / logos on the homepage

2016-07-04 Thread Kostas Tzoumas
I'd be happy to check with some of these folks On Mon, Jul 4, 2016 at 8:15 AM, Stephan Ewen wrote: > We have the "Powered By" page already. > My naive assumption was that companies that have clearance to put their > name there are also okay with a logo. > After all, it is only displaying the sam

Re: Iteration Intermediate Output

2016-05-30 Thread Kostas Tzoumas
Thanks Greg for opening this discussion! I really really don't want to derail the discussion here, just a quick clarification regarding Suneel's last email: folks that are working at data Artisans are participating in this community as individuals, not as a corporation, and the dev list is not a s

Re: Blogpost on Flink's SQL support

2016-05-24 Thread Kostas Tzoumas
+1, great post On Sun, May 22, 2016 at 4:15 PM, Matthias J. Sax wrote: > Will be a nice post! > > On 05/21/2016 10:40 PM, Henry Saputra wrote: > > I agree with Ufuk, that this is more internal posts which perfect for > blog. > > > > For high level and use cases I think would be better to be adde

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-05-16 Thread Kostas Tzoumas
+1 to Henry's comment, once this makes it to the wiki/website the wording needs to make it clear that the governance model is unchanged On Mon, May 16, 2016 at 10:02 AM, Theodore Vasiloudis < theodoros.vasilou...@gmail.com> wrote: > I like the idea of having maintainers as well, hopefully we can

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-05-13 Thread Kostas Tzoumas
Should we also add a component "Flink website and wiki" (minus the documentation) with an associated maintainer? On Fri, May 13, 2016 at 12:17 PM, Timo Walther wrote: > +1 for from my side too > > > > On 13.05.2016 06:13, Chiwan Park wrote: > >> +1 for this proposal >> > > >

Re: [PROPOSAL] Structure the Flink Open Source Development

2016-05-12 Thread Kostas Tzoumas
Big +1 from my side, I think this will help the community grow and prosper big time! On Thu, May 12, 2016 at 1:27 PM, Matthias J. Sax wrote: > +1 from my side. > > Happy to be the maintainer for Storm-Compatibiltiy (at least I guess > it's me, even the correct spelling would be with two 't' :P)

[ANNOUNCE] Flink 1.0.0 has been released

2016-03-08 Thread Kostas Tzoumas
Hi everyone! As you might have noticed, Apache Flink 1.0.0 has been released and announced! You can read more about the release at the ASF blog and the Flink blog - https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces88 - http://flink.apache.org/news/2016/03/08/relea

Re: [ANNOUNCE] Chengxiang Li added as committer

2016-01-19 Thread Kostas Tzoumas
eased to have you on board > > > On 19 Jan 2016, at 13:16, Matthias J. Sax wrote: > > > > Congrats and welcome Chengxiang!! :) > > > > On 01/19/2016 12:56 PM, Kostas Tzoumas wrote: > >> Welcome Chengxiang!! > >> > >> On Tue, Jan 19, 2016 a

Re: [ANNOUNCE] Chengxiang Li added as committer

2016-01-19 Thread Kostas Tzoumas
Welcome Chengxiang!! On Tue, Jan 19, 2016 at 12:31 PM, Stephan Ewen wrote: > Good to have you on board! > > On Tue, Jan 19, 2016 at 11:29 AM, Maximilian Michels > wrote: > > > Pleased to have you with us Chengxiang! > > > > Cheers, > > Max > > > > On Tue, Jan 19, 2016 at 11:13 AM, Chiwan Park

Re: Add CEP library to Flink

2016-01-08 Thread Kostas Tzoumas
This is a very comprehensive document, incredible job! It seems that most of the machinery is already in place in Flink, which would make this a very valuable addition taking into account the implementation effort. On Fri, Jan 8, 2016 at 3:54 PM, Till Rohrmann wrote: > Hi everybody, > > recent

Re: Dripping the Flink-on-Tez code for Flink 1.0

2016-01-08 Thread Kostas Tzoumas
for clarification, I was talking about dropping the code, I am unsure about the consequences of dripping code :-) On Fri, Jan 8, 2016 at 4:57 PM, Kostas Tzoumas wrote: > +1 from my side > > Flink on Tez never got a lot of user traction. It served well as a > prototype of "this

Re: Dripping the Flink-on-Tez code for Flink 1.0

2016-01-08 Thread Kostas Tzoumas
+1 from my side Flink on Tez never got a lot of user traction. It served well as a prototype of "this is possible", but since the core functionality will be subsumed by making Flink on YARN resource elastic, I don't see any reason we should have it as part of the Flink codebase. Best, Kostas On

Re: Effort to add SQL / StreamSQL to Flink

2016-01-07 Thread Kostas Tzoumas
Wow! Thanks Fabian, this looks fantastic! On Thu, Jan 7, 2016 at 4:35 PM, Stephan Ewen wrote: > Super, thanks for that detailed effort, Fabian! > > On Thu, Jan 7, 2016 at 3:40 PM, Matthias J. Sax wrote: > > > Pretty cool! > > > > On 01/07/2016 03:05 PM, Fabian Hueske wrote: > > > Hi everybody,

Re: [DISCUSS] Time Behavior in Streaming Jobs (Event-time/processing-time)

2015-12-17 Thread Kostas Tzoumas
Aljoscha, thanks for starting this discussion. I think this will be very important to get right. Can you explain a bit more why the results are "wrong"? I understand that window panes are built on event timestamps (as intended), but fired at regular intervals instead of watermarks. Why is this wro

[DISCUSS] Flink roadmap for 2016

2015-12-16 Thread Kostas Tzoumas
Hi everyone, I think it is very interesting to both developers and users of Flink to define a roadmap for future development. Together with Stephan, we started a draft containing a couple of areas that we think are important to focus on next. https://docs.google.com/document/d/1ExmtVpeVVT3TIhO1Jo

Re: [DISCUSS] Improving State/Timers/Windows

2015-12-14 Thread Kostas Tzoumas
> On Mon, Dec 14, 2015 at 11:18 AM, Aljoscha Krettek > > wrote: > > > >> Yes, as Kostas said, it would initially nor provide more functionality > but > >> it would enable us to add it later. > >> > >> On a side not, why would you call it KvState?

Re: [DISCUSS] Improving State/Timers/Windows

2015-12-14 Thread Kostas Tzoumas
I suppose that they can start as sugar and evolve to a different implementation. I would +1 the name change to KVState, OperatorState is indeed somewhat confusing, and it will only get harder to rename later. On Mon, Dec 14, 2015 at 11:09 AM, Gyula Fóra wrote: > Would the Reducing/Folding state

Community choice for Hadoop Summit Europe 2016

2015-12-09 Thread Kostas Tzoumas
Hi everyone, Just a reminder, the community vote for the Hadoop Summit Europe 2016 talks in Dublin is still open until December 15. There is a very good number of talks around Flink submitted, here are the ones that mention "flink" in their abstract: https://hadoopsummit.uservoice.com/search?filt

Re: Distributed DataFrame - ddf.io

2015-12-03 Thread Kostas Tzoumas
Hi Nam-Luc, I cc Rohit who implemented the DDF framework. I would say that the main difference with the Table API is that DDF aims at portability (running the same code using Flink, Spark, or a database), whereas the Table API is meant to be part of Flink itself. Best, Kostas On Thu, Dec 3, 20

Re: From 0.10 to 1.0

2015-10-23 Thread Kostas Tzoumas
+1 for 1.0, it's the right time if not a bit overdue in my opinion On Fri, Oct 23, 2015 at 12:06 PM, Fabian Hueske wrote: > Yes, let's do it > +1 > > 2015-10-23 12:00 GMT+02:00 Stephan Ewen : > > > +1 for 1.0 :-) > > > > On Fri, Oct 23, 2015 at 11:59 AM, Maximilian Michels > > wrote: > > > > >

Re: Powered by Flink

2015-10-19 Thread Kostas Tzoumas
to that page? > > Cheers, Fabian > > 2015-10-19 14:50 GMT+02:00 Kostas Tzoumas : > >> Hi everyone, >> >> I started a "Powered by Flink" wiki page, listing some of the >> organizations that are using Flink: >> >> https://cwiki.apache.org/confluence/

Powered by Flink

2015-10-19 Thread Kostas Tzoumas
Hi everyone, I started a "Powered by Flink" wiki page, listing some of the organizations that are using Flink: https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink If you would like to be added to the list, just send me a short email with your organization's name and a description

Re: An update on the DataStream API refactoring WiP

2015-10-02 Thread Kostas Tzoumas
right, I meant DataStream On Fri, Oct 2, 2015 at 2:47 PM, Robert Metzger wrote: > I suspect: "- Deletion of "DataSet.forward() and .global()"" is a typo, you > meant DataStream ? > > On Fri, Oct 2, 2015 at 2:44 PM, Kostas Tzoumas > wrote: > > >

Re: An update on the DataStream API refactoring WiP

2015-10-02 Thread Kostas Tzoumas
Oh, and of course, support for event time. I might be forgetting more, feel free to add to the list On Fri, Oct 2, 2015 at 2:40 PM, Kostas Tzoumas wrote: > Hi folks, > > Currently, Aljoscha, Stephan, and I are reworking the DataStream API as > discussed before. Things are a bit in-

An update on the DataStream API refactoring WiP

2015-10-02 Thread Kostas Tzoumas
Hi folks, Currently, Aljoscha, Stephan, and I are reworking the DataStream API as discussed before. Things are a bit in-flight right now with several commits and pull requests, and the current master containing code from both the old and the new API. I want to give you an idea of how the new API

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Kostas Tzoumas
e how people are seeing >>> Flink: It doesn't feel like second class when adding >>> "flink-streaming-core" >>> to the dependencies to me. >>> Also, the "flink-datastream-scala" module would depend on >>> "flink-dataset-scala&q

Re: Pulling Streaming out of staging and project restructure

2015-10-01 Thread Kostas Tzoumas
+1 I wanted to suggest that we rename modules to fully accept streaming as first class, qualifying also "batch" as "batch" (e.g., flink-java --> flink-dataset-java, flink-streaming --> flink-datastream, etc). This would break maven dependencies (temporary hell :-) so it's not a decision to take l

Re: Graduating Gelly in 0.10

2015-09-30 Thread Kostas Tzoumas
e is skipped. > > -V. > > On 30 September 2015 at 10:44, Ufuk Celebi wrote: > > > > > > On 30 Sep 2015, at 10:06, Kostas Tzoumas wrote: > > > > > > Does the workaround work, or can FLINK-2443 still appear? > > > > > > If we have a worka

Re: Graduating Gelly in 0.10

2015-09-30 Thread Kostas Tzoumas
Does the workaround work, or can FLINK-2443 still appear? If we have a workaround, we can schedule a fix to FLINK-2443 for 0.10.1 On Tue, Sep 29, 2015 at 9:59 PM, Vasiliki Kalavri wrote: > Hello squirrels, > > since it was already mentioned in the other thread, I'm starting this > discussion to

Release Flink 0.10

2015-09-29 Thread Kostas Tzoumas
Hi everyone, I would like to propose to cancel the 0.10-milestone release and go directly for a 0.10 release as soon as possible. My opinion would be to focus this release on: - Graduating the streaming API out of staging (depends on some open pull requests) - Master high availability - New monit

Re: Extending and improving our "How to contribute" page

2015-09-23 Thread Kostas Tzoumas
Big +1. For (1), a discussion in JIRA would also be an option IMO For (2), let us come up with few examples on what constitutes a feature that needs a design doc, and what should be in the doc (IMO architecture/general approach, components touched, interfaces changed) On Wed, Sep 23, 2015 at 2

Re: On integrating Flink with Apache NiFi

2015-09-22 Thread Kostas Tzoumas
I had a discussion with Joe from the NiFi community, and they are interested in contributing a connector between NiFi and Flink. I created a JIRA issue for that: https://issues.apache.org/jira/browse/FLINK-2740 I believe that this is the easiest and most useful integration point to begin with, as

[jira] [Created] (FLINK-2740) Create data consumer for Apache NiFi

2015-09-22 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-2740: - Summary: Create data consumer for Apache NiFi Key: FLINK-2740 URL: https://issues.apache.org/jira/browse/FLINK-2740 Project: Flink Issue Type: New Feature

Re: Releasing 0.10.0-milestone1

2015-09-08 Thread Kostas Tzoumas
+1 for a milestone release On Tue, Sep 8, 2015 at 5:43 PM, Robert Metzger wrote: > +1 for a "milestone1" release. We have a lot of good features in master > that people can benefit from. > > On Tue, Sep 8, 2015 at 5:10 PM, Maximilian Michels wrote: > > > +1 for releasing a milestone release soo

Volunteers needed for Flink Forward 2015 (and they get a free ticket)

2015-09-07 Thread Kostas Tzoumas
Hi folks, The Flink Forward 2015 organizers are looking for volunteers (and they are offering free tickets in exchange). Sign up here if you are interested (or send me an email): http://flink-forward.org/?page_id=495 Best, Kostas

[ANNOUNCE] Welcome Matthias Sax as new committer

2015-09-02 Thread Kostas Tzoumas
The Project Management Committee (PMC) of Apache Flink has asked Matthias Sax to become a committer, and we are pleased to announce that he has accepted. Matthias has been very active with Flink, and he is the original contributor of the Storm compatibility functionality. Being a committer enable

Re: 答复: Some questions about storm metrics

2015-09-02 Thread Kostas Tzoumas
Sounds great. It is easy to create an AverageAccumulator by implementing the org.apache.flink.api.common.accumulators.Accumulator interface, accumulating the sum and the count internally, and dividing in the end. This would be something useful also outside of Storm compatibility. Then, MultiReduc

Re: 答复: Some questions about storm metrics

2015-09-01 Thread Kostas Tzoumas
Flink accumulators are documented here: https://ci.apache.org/projects/flink/flink-docs-master/apis/programming_guide.html#accumulators--counters I think Aljoscha's idea is to implement Storm metrics on top of those. Huangwei, perhaps you can look whether the interfaces match (basically if Flink

Re: [DISCUSSION] Release current master as 0.9.1 (mod few changes)

2015-08-26 Thread Kostas Tzoumas
+1 for Robert's proposal On Wed, Aug 26, 2015 at 2:48 PM, Fabian Hueske wrote: > I'm +1 for Robert's proposal as well. > > 2015-08-26 14:46 GMT+02:00 Ufuk Celebi : > > > +1 > > > > I very much like Robert's suggestion. This way we can proceed with the > > 0.9.1 release as planned for the remain

[ANNOUNCE] Flink Forward 2015 program is online

2015-08-25 Thread Kostas Tzoumas
Hi everyone, Just a shoutout that we have posted the program of Flink Forward 2015 here: http://flink-forward.org/?post_type=day You can expect few changes here and there, but the majority of the talks is in. Thanks again to the speakers and the reviewers! If you have not registered yet, now is

Flink color scheme

2015-08-23 Thread Kostas Tzoumas
Hi folks, I have a color scheme for Flink that people can use for presentations, blog posts, etc, based on the Flink logo colors: https://www.dropbox.com/sh/dlstvzw2xzt09hx/AADpzAAmVUuAunWR2RJh7zjYa?dl=0 I'm not saying that we have to use it, just something that is out there in case someone want

Re: Gelly Blog Post

2015-08-23 Thread Kostas Tzoumas
I think it reads very well, time to publish :-) On Sun, Aug 23, 2015 at 12:37 PM, Martin Junghanns wrote: > Hi, > > this is a very nice blog post! I added some minor comments. I am really > excited about the future work on partition-centric computation and graph > partitioning! > > And thanks fo

Re: [ANNOUNCE] New Committer Chesnay Schepler

2015-08-20 Thread Kostas Tzoumas
Welcome Chesnay! On Thu, Aug 20, 2015 at 11:18 AM, Robert Metzger wrote: > The Project Management Committee (PMC) for Apache Flink has asked Chesnay > Schepler to become a committer and we are pleased to announce that they > have accepted. > > Chesnay has been very involved with the Flink projec

Re: DataStream API - What does "Beta" mean?

2015-08-20 Thread Kostas Tzoumas
Stephan was very fast :-) I will just add that production readiness has a lot to do with the runtime code, which is largely shared by the DataSet and the Data Stream API. The beta status does not refer to production-readiness, but rather the ability to still make breaking changes. And the roadmap

Re: [NOTICE] Reporting and fixing minor issues

2015-08-18 Thread Kostas Tzoumas
big +1 On Tue, Aug 18, 2015 at 10:43 AM, Till Rohrmann wrote: > +1, there is no point in arguing with Knuth. > > On Mon, Aug 17, 2015 at 1:07 AM, Henry Saputra > wrote: > > > +1 as well. > > > > This is a great follow-up from my previous email about adding details > > in JIRA, which also being

Re: Revert 78fd2146dd until we have consensus for FLINK-2419?

2015-07-28 Thread Kostas Tzoumas
expected in this situation. > > Well, we get to define what is expected, that's the fun of being open source :-) In my opinion it is better to provide a well tested fix later than a potentially sloppy fix earlier. > Gyula > > > Kostas Tzoumas ezt írta (időpont: 2015. júl. 2

Re: Revert 78fd2146dd until we have consensus for FLINK-2419?

2015-07-28 Thread Kostas Tzoumas
I am not familiar with this part of the code, but this is perhaps a good thing, as this is a matter of policy, not who introduced which bug (I suspect that the policy issue was Robert's motivation for starting a thread at the dev list) So, I think we have two issues: (1) Pull request https://gith

Re: Revert 78fd2146dd until we have consensus for FLINK-2419?

2015-07-28 Thread Kostas Tzoumas
I'm probably lacking a bit of context, but by reading your conversation at JIRA it seems to me that commit https://github.com/apache/flink/commit/78fd2146dd00da1130910d9f23f09e2504854ef7 does not contain a test, and Robert is asking for a test which means that we do not have consensus. If this wa

Re: Flink contributor list

2015-07-27 Thread Kostas Tzoumas
> > We could then easily add more people to the list after they > contributed. > > > > > > Cheers, > > > Max > > > > > > > > > On Fri, Jul 24, 2015 at 1:06 PM, Mingliang Qi > wrote: > > > > > > > Hey Kostas, > >

Flink contributor list

2015-07-24 Thread Kostas Tzoumas
Richter Timo Walther Maximilian Michels Andra Lungu Henry Saputra Markus Holzemer Yingjun Wu Rico Bergmann Chesnay Schepler Minglian Qi Judit Feher Jonas Traub Mathias Peters Bjorn Lohrmann Peter Szabo Alexander Alexandrov Sebastian Ruf Dennis Schneider Kostas Tzoumas Chiwan Park Paris Carbone Eric

Re: Any planning for Apache Flink 1.0 release?

2015-07-23 Thread Kostas Tzoumas
I agree that once DataStream is out of beta and stabilized we should move to 1.0. My opinion would be to aim for a 0.10 release that contains the breaking changes in the DataStream API to give people time to forward-fit applications, and go for 1.0 afterwards focusing on stability without new feat

Re: New Flink Runtime Monitor Dashboard

2015-07-22 Thread Kostas Tzoumas
Finally, Flink is a JavaScript project :-) On Wed, Jul 22, 2015 at 6:22 PM, Stephan Ewen wrote: > Hi all! > > Yesterday, we merged the framework for the new Flink runtime monitor > dashboard. New stub, based on angular.js and coffee script. > > https://github.com/apache/flink/tree/master/flink-r

Re: Theoretical complexity of a coGroup

2015-07-22 Thread Kostas Tzoumas
Hi Andra, CoGroup is not implemented as a Cartesian product, so O(V*E) is not a very accurate approximation. All this depends on what you count. Let's assume single-node execution and that everything fits in memory, and let's count comparisons and UDFs on groups. Then, coGroup sorts both inputs,

Re: Read XML from HDFS

2015-07-15 Thread Kostas Tzoumas
Perhaps there is also an existing HadoopInputFormat for XML that you might be able to reuse for your purposes (Flink supports Hadoop input formats). For example, there is an XMLInputFormat in the Apache Mahout codebase that you could take a look at: https://github.com/apache/mahout/blob/ad84344e40

Re: Student looking to contribute to Stratosphere

2015-07-15 Thread Kostas Tzoumas
gt; On Wed, Jul 15, 2015 at 4:44 PM, Kostas Tzoumas > wrote: > > > Hi Rohit, > > > > If you are just working on your laptop, I personally find it much easier > to > > work without Hadoop and use the local file system or just Java > collections > > for

Re: Student looking to contribute to Stratosphere

2015-07-15 Thread Kostas Tzoumas
Hi Rohit, If you are just working on your laptop, I personally find it much easier to work without Hadoop and use the local file system or just Java collections for testing and trying out ideas. When you move to a cluster, it is common to use a Hadoop installation to store large files in HDFS. Th

Re: Design documents for consolidated DataStream API

2015-07-14 Thread Kostas Tzoumas
> > > change names easily (before the release). > > > >> > > > > > > > > >> > > > > > In fact, I don't think it leaves a bad impression. Global > > > >> windows > > > >> > are

Re: Design documents for consolidated DataStream API

2015-07-13 Thread Kostas Tzoumas
+1 from my side On Mon, Jul 13, 2015 at 4:15 PM, Stephan Ewen wrote: > Do we have consensus on these designs? > > If we have, we should get to implementing this soon, because basically all > streaming patches will have to be revisited in light of this... > > On Tue, Jul 7, 2015 at 3:41 PM, Gyula

Re: Does DataSet job also use Barriers to ensure "exactly once."?

2015-07-09 Thread Kostas Tzoumas
No, it doesn't; periodic snapshots are not needed in DataSet programs, as DataSets are of finite size and failed partitions can be replayed completely. On Thu, Jul 9, 2015 at 2:43 PM, 马国维 wrote: > hi, everyoneThe doc say Flink Streaming use "Barriers" to ensure > "exactly once."Does the DataSe

Re: Flink on Wikipedia

2015-07-07 Thread Kostas Tzoumas
I think it is clear to most people that the only official and (hopefully) up-to-date description of an Apache project is its Apache website, and any paper can get outdated. Perhaps we can change the link to a more up-to-date paper when we have one. I like the article, thanks Matthias! Kostas On

Re: Rework of streaming iteration API

2015-07-07 Thread Kostas Tzoumas
I see. Perhaps more important IMO is defining the semantics of stream loops with event time. The reason I asked about nested is that Naiad and other designs used a multidimensional timestamp to capture loops: (outer loop counter, inner loop counter, timestamp). I assume that currently making sense

Re: Rework of streaming iteration API

2015-07-07 Thread Kostas Tzoumas
+1 for rethinking the iterations in DataStream However, wouldn't this proposal allow the definition of arbitrary loops (e.g., nested loops) that are not well behaved afaik? On Tue, Jul 7, 2015 at 4:12 PM, Stephan Ewen wrote: > I see that the newly proposed API makes some things easier to define

Re: Thoughts About Streaming

2015-06-24 Thread Kostas Tzoumas
I agree with supporting out-of-order out of the box :-), even if this means a major refactoring. This is the right time to refactor the streaming API before we pull it out of beta. I think that this is more important than new features in the streaming API, which can be prioritized once the API is o

Re: Send events to parallel operator instances

2015-06-04 Thread Kostas Tzoumas
Wouldn't this kind of cross-task communication break the whole dataflow abstraction? How can recovery be implemented if we allowed something like this? On Thu, Jun 4, 2015 at 5:14 PM, Stephan Ewen wrote: > That is not what Ufuk said. You can use a singleton auxiliary task that > communicates in

Re: Changed the behavior of "DataSet.print()"

2015-06-02 Thread Kostas Tzoumas
+1 for printOnTaskManager(prefix) On Tue, Jun 2, 2015 at 1:35 PM, Till Rohrmann wrote: > +1 for printOnTaskManager(prefix) > > On Tue, Jun 2, 2015 at 12:08 PM, Fabian Hueske wrote: > > > +1 for writeToWorkerStdOut(prefix) > > On Jun 2, 2015 11:42, "Aljoscha Krettek" wrote: > > > > > +1 for pri

Re: SQL on Flink

2015-05-27 Thread Kostas Tzoumas
very excited to see this starting! On Wed, May 27, 2015 at 6:06 PM, Ufuk Celebi wrote: > > On 27 May 2015, at 17:05, Timo Walther wrote: > > > It's rather passion for the future of the project than passion for SQL > ;-) > > > > I always try to think like someone from the economy. And IMO the gu

Re: SQL on Flink

2015-05-27 Thread Kostas Tzoumas
I think Fabian's arguments make a lot of sense. However, if Timo *really wants* to start SQL on top of Table, that is what he will do a great job at :-) As usual, we can keep it in beta status in flink-staging until it is mature... and it will help create issues for the Table API and give directio

Re: Gelly Roadmap

2015-05-20 Thread Kostas Tzoumas
:-D Great! On Tue, May 19, 2015 at 4:00 PM, Andra Lungu wrote: > Hi Kostas, > > We're way ahead of you! The first draft of the blog post is internally > reviewed as we speak ;) > > > On Tue, May 19, 2015 at 3:49 PM, Kostas Tzoumas > wrote: > > > This is

Re: Gelly Roadmap

2015-05-19 Thread Kostas Tzoumas
This is very cool! Would also love to see a blog post on Gelly at some point, perhaps finishing off with the roadmap On Tue, May 19, 2015 at 10:55 AM, Andra Lungu wrote: > Apache TinkerPop made it to the wish-list :) > Thanks, Fabian! > > On Tue, May 19, 2015 at 9:42 AM, Flavio Pompermaier > w

Re: New project website

2015-05-15 Thread Kostas Tzoumas
+1 On Fri, May 15, 2015 at 11:49 AM, Vasiliki Kalavri < vasilikikala...@gmail.com> wrote: > +1 :)) > > On 15 May 2015 at 12:42, Ufuk Celebi wrote: > > > > > On 14 May 2015, at 12:39, Vasiliki Kalavri wrote: > > > > > Hey Ufuk, > > > > > > the logo still looks too big for the menu and so does th

Re: New project website

2015-05-12 Thread Kostas Tzoumas
Good points raised by Stephan, Felix, and Volker. Do you think we can achieve this by iterating on the new design? One thing we can indeed do is de-clutter a bit. Apart from that, what high level points would you like to see in the frontpage? On Tue, May 12, 2015 at 6:16 AM, Markl, Volker, Prof.

Re: [DISCUSS] Change Streaming Operators to be Push-Only

2015-05-05 Thread Kostas Tzoumas
operator, then they all have to manage input > readers and inline events manually. > > On Tue, May 5, 2015 at 2:41 PM, Kostas Tzoumas > wrote: > > Can you give us a rough idea of the pros and cons? Do we lose some > > functionality by getting rid of iterations? > > &

Re: [DISCUSS] Change Streaming Operators to be Push-Only

2015-05-05 Thread Kostas Tzoumas
Can you give us a rough idea of the pros and cons? Do we lose some functionality by getting rid of iterations? Kostas On Tue, May 5, 2015 at 1:37 PM, Aljoscha Krettek wrote: > Hi Folks, > while working on introducing source-assigned timestamps into streaming > (https://issues.apache.org/jira/br

Re: Adding a new operator

2015-04-27 Thread Kostas Tzoumas
Some form of tree aggregation is useful in many cases, and IMO a good addition to the system. Kostas On Mon, Apr 27, 2015 at 11:04 AM, Andra Lungu wrote: > Hi Fabian, > > After a quick look at the current behaviour of Flink's combinable reduce, I > saw that it does something like this: > > http

Re: About Operator and OperatorBase

2015-04-22 Thread Kostas Tzoumas
I think Stephan meant Meteor, an old API when Flink was Stratosphere. This was never part of the code that made it to Apache. Not sure if we want to remove the common API, as it provides a dataflow abstraction that is higher level than the JobGraph. Admittedly, I don't have a better argument other

Re: [DISCUSS] Replace user@f.a.o with Stackoverflow

2015-04-20 Thread Kostas Tzoumas
If we clearly preface with a boilerplate text and a link to the users@ question it might look less weird :-) Perhaps we can create a stackoverflow user whose purpose is to create questions that mirror the users@ list? Not sure how much this can be automated though On Mon, Apr 20, 2015 at 4:25 PM,

Re: Merge Python API

2015-04-20 Thread Kostas Tzoumas
I'm +1 for this On Mon, Apr 20, 2015 at 11:03 AM, Robert Metzger wrote: > Hi, > > The Python API pull request [1] has been open for quite some time now. > > I was wondering whether we are planning to merge it or not. > I took a closer look at the Python API a few weeks ago and I think we > shoul

[jira] [Created] (FLINK-1898) Add support for self-joins to Flink on Tez

2015-04-16 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-1898: - Summary: Add support for self-joins to Flink on Tez Key: FLINK-1898 URL: https://issues.apache.org/jira/browse/FLINK-1898 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-1897) Add accummulators and counters feature to Flink on Tez

2015-04-16 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-1897: - Summary: Add accummulators and counters feature to Flink on Tez Key: FLINK-1897 URL: https://issues.apache.org/jira/browse/FLINK-1897 Project: Flink Issue

[jira] [Created] (FLINK-1895) Add task chaining to Flink on Tez

2015-04-16 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-1895: - Summary: Add task chaining to Flink on Tez Key: FLINK-1895 URL: https://issues.apache.org/jira/browse/FLINK-1895 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-1896) Add broadcast variables feature to Flink on Tez

2015-04-16 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-1896: - Summary: Add broadcast variables feature to Flink on Tez Key: FLINK-1896 URL: https://issues.apache.org/jira/browse/FLINK-1896 Project: Flink Issue Type

[jira] [Created] (FLINK-1894) Add Tez execution mode to Flink command-line tools

2015-04-16 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-1894: - Summary: Add Tez execution mode to Flink command-line tools Key: FLINK-1894 URL: https://issues.apache.org/jira/browse/FLINK-1894 Project: Flink Issue

[jira] [Created] (FLINK-1893) Add Scala support for Flink on Tez

2015-04-16 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-1893: - Summary: Add Scala support for Flink on Tez Key: FLINK-1893 URL: https://issues.apache.org/jira/browse/FLINK-1893 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-1892) Local job execution does not exit.

2015-04-16 Thread Kostas Tzoumas (JIRA)
Kostas Tzoumas created FLINK-1892: - Summary: Local job execution does not exit. Key: FLINK-1892 URL: https://issues.apache.org/jira/browse/FLINK-1892 Project: Flink Issue Type: Bug

Re: Flink interactive Scala shell

2015-04-16 Thread Kostas Tzoumas
Great, let us know if you run into any issues. Can you create a JIRA on the REPL and link to your repository for the community to track the status? On Wed, Apr 15, 2015 at 4:23 PM, Nikolaas s wrote: > Thanks for the feedback guys! > Apparently The Scala Shell compiles the Shell input to some ki

Re: Flink 0.9.0-milestone1 released

2015-04-13 Thread Kostas Tzoumas
should be fixed now thanks! On Mon, Apr 13, 2015 at 5:48 PM, Vasiliki Kalavri wrote: > Awesome job everyone! > > Just one thing, the release download link in the announcement post is > broken :-) > > Looking forward to 0.9.0 ^^ > > Cheers, > -V. > > On 13 Ap

Flink 0.9.0-milestone1 released

2015-04-13 Thread Kostas Tzoumas
We are very excited to announce Flink 0.9.0-milestone1, a preview release to give users early access to some Flink 0.9.0 features, including: - A Table API for SQL-like queries embedded in Java and Scala - Gelly, Flink's graph processing API - A Machine Learning library on Flink inspired by scikit

Re: [VOTE] Release Apache Flink 0.9.0-milestone-1 (RC1)

2015-04-09 Thread Kostas Tzoumas
+1 Ran tests on a debian machine. Ran examples on a 4-node cluster via the YARN client. On Thu, Apr 9, 2015 at 12:15 PM, Ufuk Celebi wrote: > +1 > > - Checked Hashes and signing > - Ran on 25 node cluster and checked locality of input split assignment > - Build from source and ran all tests >

Flink Forward 2015

2015-04-07 Thread Kostas Tzoumas
Hi everyone, The folks at data Artisans and the Berlin Big Data Center are organizing the first physical conference all about Apache Flink in Berlin the coming October: http://flink-forward.org The conference will be held in a beautiful spot an old brewery turned event space (the same space that

  1   2   >