[jira] [Created] (FLINK-1368) Change memory management settings for Streaming programs

Gyula Fora created FLINK-1368: - Summary: Change memory management settings for Streaming programs Key: FLINK-1368 URL: https://issues.apache.org/jira/browse/FLINK-1368 Project: Flink Issue Type

[jira] [Created] (FLINK-1367) Add field aggregations to Streaming Scala api

Gyula Fora created FLINK-1367: - Summary: Add field aggregations to Streaming Scala api Key: FLINK-1367 URL: https://issues.apache.org/jira/browse/FLINK-1367 Project: Flink Issue Type: New

[jira] [Created] (FLINK-1364) No simple way to group on the whole Tuple/element for DataSets/Streams

Gyula Fora created FLINK-1364: - Summary: No simple way to group on the whole Tuple/element for DataSets/Streams Key: FLINK-1364 URL: https://issues.apache.org/jira/browse/FLINK-1364 Project: Flink

[jira] [Created] (FLINK-1345) Add operator chaining to the streaming runtime

Gyula Fora created FLINK-1345: - Summary: Add operator chaining to the streaming runtime Key: FLINK-1345 URL: https://issues.apache.org/jira/browse/FLINK-1345 Project: Flink Issue Type

Re: Eclipse import errors after akka update

Hey, Thanks Till for the description, I actually thought that I had the things set already what you described since I have been working on the scala code anyways. It turned out that the xplugin path self updated itself by pasting my workspace path in front of path. So when I checked if I have

Re: Streaming windowing rework

Mon, Dec 8, 2014 at 3:22 AM, Paris Carbone >>> wrote: >>>> Hey, >>>> >>>> @Gyula I can help with the discussion content doc >>>> @Gianmarco It will be great to have you! We will also keep you posted >> on >>> the Samoa integration. &

Streaming scala api

Hey, I have started developing the scala api for streaming on this git branch: https://github.com/mbalassi/incubator-flink/tree/scala-api (you can try it out) I have done some refactoring in the java api to make it easier and already

Re: Eclipse and Scala Setup Issues

+1 And I would also like to add on the Eclipse setup side that, you should need to watch out when installing the Scala IDE plugin, because the one offered in the marketplace is for 2.11. And also if you install the one for 2.10 it will say that there is an update (which again is 2.11). And this

[jira] [Created] (FLINK-1313) Add support for out-of-place aggregations for streaming

Gyula Fora created FLINK-1313: - Summary: Add support for out-of-place aggregations for streaming Key: FLINK-1313 URL: https://issues.apache.org/jira/browse/FLINK-1313 Project: Flink Issue Type

[jira] [Created] (FLINK-1312) Change OutputSelector to SAM-type to allow lambdas for split operator

Gyula Fora created FLINK-1312: - Summary: Change OutputSelector to SAM-type to allow lambdas for split operator Key: FLINK-1312 URL: https://issues.apache.org/jira/browse/FLINK-1312 Project: Flink

Re: Setting up IRC channel to promote discussions and getting help

+1 I think it’s a good idea! Gyula > On 08 Dec 2014, at 22:45, Henry Saputra wrote: > > HI All, > > Sorry for cross posting. > > I thinking about setting up ASF official IRC channel for Apache Flink. > > Some of us could just hang out and provide answer or just ask for > question about Flink

Re: Streaming windowing rework

Hey, I’m glad that you want to join. We will post a hangout link here later :) Can someone please help me with setting these things up? Cheers, Gyula > On 08 Dec 2014, at 11:16, Gianmarco De Francisci Morales > wrote: > > Hi, > I'm one of the developers of SAMOA and I'd be happy to participa

Re: Streaming windowing rework

Good for me as well :) Gyula > On 07 Dec 2014, at 21:14, Vasiliki Kalavri wrote: > > +1 I'm in for Friday too! > > On 7 December 2014 at 20:42, Márton Balassi > wrote: > >> Friday late afternoon works for me. :) >> >> On Sun, Dec 7, 2014 at 8:30 PM, Paris Carbone wrote: >> >>> +1 >>> >>>

Re: [VOTE] Graduate Flink from the Incubator

> >>>>> RESOLVED, that the Apache Flink Project be and hereby is >>>>> responsible for the creation and maintenance of software >>>>> related to fast and reliable large-scale data analysis >>>>> with focus on pro

Re: New Flink website layout and frontage

+1 I also think that at looks very good! Stylish :) > On 27 Nov 2014, at 20:32, Stephan Ewen wrote: > > I like it as well. Much better than the current website :-) > > On Thu, Nov 27, 2014 at 8:30 PM, Aljoscha Krettek > wrote: > >> The formatting of the Scala example code is a bit off in pla

Re: Channel indexing with pointwise connection pattern

modified version of the co-location constraint. > > Stephan > Am 27.11.2014 00:45 schrieb "Gyula Fora" : > >> Hey, >> >> I was hoping that someone can answer this right away without me having to >> dig through all the code :) >> >> How doe

Channel indexing with pointwise connection pattern

Hey, I was hoping that someone can answer this right away without me having to dig through all the code :) How does the channel indexing go when more then one consumer subtask is connected to an intermediate dataset in the pointwise pattern? I am trying to figure out which one is the “in-memor

[jira] [Created] (FLINK-1280) Rework StreamInvokables to use the PactDriver interface for better future integration

Gyula Fora created FLINK-1280: - Summary: Rework StreamInvokables to use the PactDriver interface for better future integration Key: FLINK-1280 URL: https://issues.apache.org/jira/browse/FLINK-1280

[jira] [Created] (FLINK-1279) Change default partitioning setting for low parallelism stream sources from forward to distribute

Gyula Fora created FLINK-1279: - Summary: Change default partitioning setting for low parallelism stream sources from forward to distribute Key: FLINK-1279 URL: https://issues.apache.org/jira/browse/FLINK-1279

Re: Question Streaming API - window(win_length,slide_step)

Hello, Nice question, honestly we haven’t thought about this functionality yet but it makes a very a good point. (I am also forwarding this to the dev-list, it’s more fitting there) In the current API there is no straightforward way of getting this information. What you could do at the moment

Re: [ANNOUNCE] Welcome Vasia Kalavri as a new committer

Congrats and welcome! :) > On 14 Nov 2014, at 14:52, Kostas Tzoumas wrote: > > Hi everyone, > > In the PPMC we have voted to add Vasia Kalavri as a new committer to Flink. > > Vasia, welcome! > > Kostas

Re: Hi / Aggregation support

I also support this approach: ds.groupBy(0).aggregate(min(1), max(1), cnt()) I think it makes the code more readable, because it is easy to see whats in the result tuple. Gyula > On 10 Nov 2014, at 10:49, Aljoscha Krettek wrote: > > I like this version: ds.groupBy(0).aggregate(min(1), max(

[jira] [Created] (FLINK-1224) Fix .getExecutionEnvironment of the StreamingExecutionEnvironment to return proper context

Gyula Fora created FLINK-1224: - Summary: Fix .getExecutionEnvironment of the StreamingExecutionEnvironment to return proper context Key: FLINK-1224 URL: https://issues.apache.org/jira/browse/FLINK-1224

Re: Making Fink More HFT / Trading Friendly

Hi, Thanks for the suggestions, some of these ideas are already present in Flink, and we will work on these for streaming as well. Gyula > On 07 Nov 2014, at 11:30, sirinath wrote: > > Hi, > > In response to the message exchanged in LinkedIn I am shooting a few ideas. > > To make Flink a gr

Re: Make Streaming Interoperable with Reactive Streams, Java Streams and Rx

Thank you for your suggestions. We are planning to extend connector support to many other systems. In the meantime, any contribution regarding this is warmly welcome. > On 07 Nov 2014, at 07:02, sirinath wrote: > > What I ment was to have stream connectors for Java Streams, Rx > (http://reactiv

Re: Make Streaming Interoperable with Reactive Streams, Java Streams and Rx

Hey Suminda, I am not 100% by what you mean by interoperating, so clarify me if I am off-track here :) The systems that you have mentioned provide programming models for stream processing systems, in that sense a lot of features of Flink runtime are similar to these systems but it is presented

Re: Streaming groupby and aggregation by field expressions

Hi Fabian, The link you sent is broken but I think you are referring the conversation with Viktor. I agree that we should provide the same api for aggregations and I don’t think there is any reason why the proposed batch approach wouldn’t work on streaming :) When there is an agreement on the

Re: how load/group with large csv files

xtractor, would I have to do the > sort myself then? > > cheers Martin > > On Tue, Oct 21, 2014 at 2:08 PM, Gyula Fora wrote: > >> Hey, >> >> Using arrays is probably a convenient way to do so. >> >> I think the way you described the groupBy only

Re: how load/group with large csv files

Hey, Using arrays is probably a convenient way to do so. I think the way you described the groupBy only works for tuples now. To do the grouping on the array field, you would need to create a key extractor for this and pass that to groupBy. Actually we have some use-cases like this for streami

[jira] [Created] (FLINK-1176) Add operator for global reduce/aggregations

Gyula Fora created FLINK-1176: - Summary: Add operator for global reduce/aggregations Key: FLINK-1176 URL: https://issues.apache.org/jira/browse/FLINK-1176 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-1175) Add support for KeySelector and possibility to select more than one field for groupBy for streaming

Gyula Fora created FLINK-1175: - Summary: Add support for KeySelector and possibility to select more than one field for groupBy for streaming Key: FLINK-1175 URL: https://issues.apache.org/jira/browse/FLINK-1175

[jira] [Created] (FLINK-1174) Add operator for temporal joins over time windows of two DataStreams

Gyula Fora created FLINK-1174: - Summary: Add operator for temporal joins over time windows of two DataStreams Key: FLINK-1174 URL: https://issues.apache.org/jira/browse/FLINK-1174 Project: Flink

[jira] [Created] (FLINK-1173) Add socket text stream as a data source for the streaming API

Gyula Fora created FLINK-1173: - Summary: Add socket text stream as a data source for the streaming API Key: FLINK-1173 URL: https://issues.apache.org/jira/browse/FLINK-1173 Project: Flink Issue

Re: Help with starting contributing to Flink

Hey, Yes, just decide what you would like to work on, and if that happens to be the Streaming API, we will be glad to assist you with your effort in any way. There is always something to do from the pretty simple to the very challenging ones so we can definitely come up with something that you w

[jira] [Created] (FLINK-1161) Modify streaming api type extractor classes to allow java 8 lambdas

Gyula Fora created FLINK-1161: - Summary: Modify streaming api type extractor classes to allow java 8 lambdas Key: FLINK-1161 URL: https://issues.apache.org/jira/browse/FLINK-1161 Project: Flink

[jira] [Created] (FLINK-1160) Modify record readers to allow publishing events to specific inputs (instead of broadcast) in the union record reader

Gyula Fora created FLINK-1160: - Summary: Modify record readers to allow publishing events to specific inputs (instead of broadcast) in the union record reader Key: FLINK-1160 URL: https://issues.apache.org/jira

Re: Eclipse issue after latest dependency rework pr

That worked, thanks. On 30 Sep 2014, at 23:12, Stephan Ewen wrote: > Yes, I wanted to write a few pointers about such issues. > > We recently shaded the guava dependency, meaning that we have a custom > version of guava where all classes reside in > "org.apache.flink.shaded.com.google" and mave

Re: Planning Flink release 0.7-incubating

Hey, So what is the current decision regarding the time of the upcoming release? As for the streaming component, we included all the features we wanted, we will start to test everything tomorrow, making sure that all works as intended. We are also almost finished with cleaning up the connector

Re: Clean up dependencies in streaming connectors

Thanks, I will look into this and try to figure it out, as you can see I am not a maven pro :) On 29 Sep 2014, at 18:44, Stephan Ewen wrote: > You may be able to solve this with careful exclusions. > > It seems kafka is monolithic, having no separation between connector and > engine. If you kn

[jira] [Created] (FLINK-1121) Add minBy and maxBy operators to the streaming api

Gyula Fora created FLINK-1121: - Summary: Add minBy and maxBy operators to the streaming api Key: FLINK-1121 URL: https://issues.apache.org/jira/browse/FLINK-1121 Project: Flink Issue Type

Re: [VOTE] Style of Flink squirrel logo for website and public accounts

+COLORED On 18 Sep 2014, at 12:28, Fabian Hueske wrote: > Please vote on which style of Flink's squirrel logo should be used for > Flink's website and other public accounts (e.g., Twitter). > > The logo comes in three styles: > - COLORED (in Apache colors) > - WHITE (white logo on dark backgrou

Re: Flink Hackathon in Stockholm (October 8th / 9th)

eneral improvements with/to the experimental streaming API >> > >>> - Graph Processing - Brain storming and hacking on a prototype of the Graph >>> processing DSL >> > >>> So far, we have as topic-leaders / participants: >> > >>> S

Re: [DISCUSS] Logo for Flink

+1 for all of them Thank you very much, Linda! On 16 Sep 2014, at 21:47, Ufuk Celebi wrote: > Very nice work! :) I would also like to use the right one in the right > context. > > On Tuesday, September 16, 2014, Aljoscha Krettek > wrote: > >> Do we have to vote on the Variations? Couldn't we

[jira] [Created] (FLINK-1081) Add HDFS file-stream source for streaming

Gyula Fora created FLINK-1081: - Summary: Add HDFS file-stream source for streaming Key: FLINK-1081 URL: https://issues.apache.org/jira/browse/FLINK-1081 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-1021) IllegalStateException at InputGate

Gyula Fora created FLINK-1021: - Summary: IllegalStateException at InputGate Key: FLINK-1021 URL: https://issues.apache.org/jira/browse/FLINK-1021 Project: Flink Issue Type: Bug