Re: Parallelism vs task manager allocation

2016-09-20 Thread pushpendra.jaiswal
Hi Greg My sink is a hashmap(acting as db store), I am reading from this hashmap from other query stream. I need one instance / slot per task manager of sink, so that everyone refers the same instance/slot. So is there any way so that I can restrict 1 slot per task manager. Thanks Pushpendra Jais

Re: Sharing Java Collections within Flink Cluster

2016-09-07 Thread pushpendra.jaiswal
Hi Fabian I am also looking for this solution, could you help me with two things: 1. How this is different from Queryable state. 2. How to query this key-value state from DS2 even if its running in the same application. e.g. val keyedStream = stream.keyby(_.key) val otherStream = somekafka.cr

Re: assignTimestamp after keyBy

2016-09-07 Thread pushpendra.jaiswal
Please refer https://ci.apache.org/projects/flink/flink-docs-master/dev/event_timestamps_watermarks.html for assigning timestamps. You can do map after keyby to assign timestamps e.g: val withTimestampsAndWatermarks: DataStream[MyEvent] = stream .filter( _.severity == WARNING )

Re: Not able to query : Queryable State

2016-09-06 Thread pushpendra.jaiswal
Hi Stefan Please find below stack trace and code : java.lang.IllegalStateException: Job 81ca41b13e7be8feb99f064e5a9a4237 not found at org.apache.flink.runtime.jobmanager.JobManager.org$apache$flink$runtime$jobmanager$JobManager$$handleKvStateMessage(JobManager.scala:1470) at org.a

Re: Not able to query : Queryable State

2016-09-06 Thread pushpendra.jaiswal
Hi Vishnu /val env = StreamExecutionEnvironment.getExecutionEnvironment val jobID = env.getStreamGraph.getJobGraph.getJobID/ As I am using the jobId of current running job. It should exist. Thanks and regards Pushpendra -- View this message in context: http://apache-flink-user-mailing-list-