a question about operatorChain.broadcastCheckpointBarrier

2016-03-09 Thread Ma GuoWei
hi,all Why does the broadcastCheckpointBarrier need in the lock scope? thanks a lot. [cid:3612d450-1757-4dcb-9bb5-0d0cd8b89b1a]

Flink-1.0.0 JobManager is not running in Docker Container on AWS

2016-03-09 Thread Deepak Jha
Hi All, I'm trying to setup Flink 1.0.0 cluster on Docker (separate containers for jobmanager and taskmanager) inside AWS (Using AWS ECS service). I tested it locally and its working fine but on AWS Docker, I am running into following issue *2016-03-09 18:04:12,114 PST [INFO] ec2-52-3-248-202.co

Re: New user of Flink

2016-03-09 Thread janardhan shetty
Resolved the same by setting JAVA_HOME in intelli J. On Wed, Mar 9, 2016 at 12:27 PM, janardhan shetty wrote: > Hi Flink team, > > I am setting up my dev environment to start developing flink applications > primarily on Java. > Can you pls provide few clarifications: > > Fyi, > First time use of

Re: New user of Flink

2016-03-09 Thread Jamie Grier
Yes, in IntelliJ you just need to setup a JDK. If you don't already have one just download and install a recent JDK and then point IntelliJ to it. On Wed, Mar 9, 2016 at 12:27 PM, janardhan shetty wrote: > Hi Flink team, > > I am setting up my dev environment to start developing flink applicati

New user of Flink

2016-03-09 Thread janardhan shetty
Hi Flink team, I am setting up my dev environment to start developing flink applications primarily on Java. Can you pls provide few clarifications: Fyi, First time use of Intelli J and Flink: Clarifications needed: 1. After importing quickstart as maven project in intelli J . It asks for SDK no

[jira] [Created] (FLINK-3597) Table API operator names should reflect relational expression

2016-03-09 Thread Fabian Hueske (JIRA)
Fabian Hueske created FLINK-3597: Summary: Table API operator names should reflect relational expression Key: FLINK-3597 URL: https://issues.apache.org/jira/browse/FLINK-3597 Project: Flink

Re: Tuple performance and the curious JIT compiler

2016-03-09 Thread Stephan Ewen
Thanks for posting this. I think it is not super urgent (in the sense of weeks or few months), so results around mid summer is probably good. The background in LLVM is a very good base for this! On Wed, Mar 9, 2016 at 3:56 PM, Gábor Horváth wrote: > Hi, > > In the meantime I sent out the curren

Design Doc: Execution of Table API & SQL queries in Java/Scala programs

2016-03-09 Thread Fabian Hueske
Hi everybody, the following design document shows different ways to execute Table API and SQL queries (query in the following) in Java and Scala programs. It proposes alternatives for data sources and result sinks of queries. Data Sources - Use a DataSet, DataStream, or Table as input for a query

Re: GSoC Project Proposal Draft: Code Generation in Serializers

2016-03-09 Thread Gábor Horváth
Hi! As far as I can see the formatting was not correct in my previous mail. A better formatted version is available here: https://docs.google.com/document/d/1VC8lCeErx9kI5lCMPiUn625PO0rxR-iKlVqtt3hkVnk Sorry for that. Regards, Gábor On 9 March 2016 at 15:51, Gábor Horváth wrote: > Hi,I did not

Re: Tuple performance and the curious JIT compiler

2016-03-09 Thread Gábor Horváth
Hi, In the meantime I sent out the current version of the proposal draft [1]. Hopefully it will help you triage this task and contribute to the discussion of the problem. How urgent is this issue? In what time frame should there be results? Best Regards, Gábor [1] http://apache-flink-mailing-lis

GSoC Project Proposal Draft: Code Generation in Serializers

2016-03-09 Thread Gábor Horváth
Hi,I did not want to send this proposal out before the I have some initial benchmarks, but this issue was mentioned on the mailing list ( http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Tuple-performance-and-the-curious-JIT-compiler-td10666.html), and I wanted to make this informatio

[jira] [Created] (FLINK-3596) DataSet RelNode refactoring

2016-03-09 Thread Vasia Kalavri (JIRA)
Vasia Kalavri created FLINK-3596: Summary: DataSet RelNode refactoring Key: FLINK-3596 URL: https://issues.apache.org/jira/browse/FLINK-3596 Project: Flink Issue Type: Improvement C

Re: Tuple performance and the curious JIT compiler

2016-03-09 Thread Stephan Ewen
Do we have consensus that we want to "reserve" this topic for a GSoC student? It is becoming a feature that gains more importance. To see we can "hold off" on working on that, would be good to know a bit more, like - when is it decided whether this project takes place? - when would results be

[jira] [Created] (FLINK-3595) Kafka09 consumer thread does not interrupt when stuck in record emission

2016-03-09 Thread Stephan Ewen (JIRA)
Stephan Ewen created FLINK-3595: --- Summary: Kafka09 consumer thread does not interrupt when stuck in record emission Key: FLINK-3595 URL: https://issues.apache.org/jira/browse/FLINK-3595 Project: Flink

Re: Accessing the Configuration

2016-03-09 Thread Márton Balassi
Thanks for the clarification. On Wed, Mar 9, 2016 at 2:00 PM, Stephan Ewen wrote: > I think that is not possible currently. Would not be too hard to give > access to the TM config via the RuntimeContext (in a similar way as the > user code classloader, for example). > > On Wed, Mar 9, 2016 at 12

Re: Accessing the Configuration

2016-03-09 Thread Stephan Ewen
I think that is not possible currently. Would not be too hard to give access to the TM config via the RuntimeContext (in a similar way as the user code classloader, for example). On Wed, Mar 9, 2016 at 12:38 PM, Márton Balassi wrote: > Hey, > > I was wondering whether there is a way to access th

Accessing the Configuration

2016-03-09 Thread Márton Balassi
Hey, I was wondering whether there is a way to access the Configuration from an (Stream)ExecutionEnviroment or a RichFunction. Practically I would like to set a temporary persist path in the Configuration and access the location somewhere during the topology. I have followed the way the streaming

Re: [ANNOUNCE] Flink 1.0.0 has been released

2016-03-09 Thread Stephan Ewen
@Igor: State from the CEP library (like what was the last observed event for a certain ID or so) is kept in Flink's state mechanisms (the key/value state) Have a look at these docs for details: - Working with State: https://ci.apache.org/projects/flink/flink-docs-release-1.0/apis/streaming/state

[jira] [Created] (FLINK-3594) Unstable KafkaConsumerTestBase

2016-03-09 Thread Chesnay Schepler (JIRA)
Chesnay Schepler created FLINK-3594: --- Summary: Unstable KafkaConsumerTestBase Key: FLINK-3594 URL: https://issues.apache.org/jira/browse/FLINK-3594 Project: Flink Issue Type: Bug