Re: Not able to query : Queryable State

2016-09-06 Thread pushpendra.jaiswal
Hi Vishnu /val env = StreamExecutionEnvironment.getExecutionEnvironment val jobID = env.getStreamGraph.getJobGraph.getJobID/ As I am using the jobId of current running job. It should exist. Thanks and regards Pushpendra -- View this message in context:

Re: Flink Iterations vs. While loop

2016-09-06 Thread Dan Drewes
Hi, I am not broadcasting the data but the model, i.e. the weight vector contained in the "State". You are right, it would be better for the implementation with the while loop to have the data on HDFS. But that's exactly the point of my question: Why are the Flink Iterations not faster if you

回复:Re: 回复:Re: modify coGroup GlobalWindows_GlobalWindow

2016-09-06 Thread rimin515
think your anwser. but i can not get your ideal."If all elements of "words2" have been processed, the right side of your coGroup will always be empty no matter what is incoming in your socketTextStream.",the mean i can not get. the following is the ideal from me(it maybe error): the

回复:Re: 回复:Re: fromParallelCollection

2016-09-06 Thread rimin515
my data from a Hbase table ,it is like a List[rowkey,Map[String,String]], class MySplittableIterator extends SplittableIterator[String]{ // Members declared in java.util.Iterator def hasNext(): Boolean = { } def next(): Nothing = { } // Members

Re: 回复:Re: modify coGroup GlobalWindows GlobalWindow

2016-09-06 Thread Timo Walther
I think you have to rethink your approach. In your example "words2" is a stream but only with a fixed set of elements. If all elements of "words2" have been processed, the right side of your coGroup will always be empty no matter what is incoming in your socketTextStream. It is not read in

Re: Stream sql query in Flink

2016-09-06 Thread Timo Walther
Hi, this looks like a bug. I created an issue for it (https://issues.apache.org/jira/browse/FLINK-4581). Could you also send us the pom.xml you are using for your project? Timo Am 06/09/16 um 13:47 schrieb jiecxy: Hi all, I want to write a program, a thread read the real-time message

回复:Re: modify coGroup GlobalWindows GlobalWindow

2016-09-06 Thread rimin515
i try read data into a list or List[Map] to store the T2,but i think if use list or List[Map],there is not parallelization,so i want to use coGroup. other hand,the coGroup function is join the T1 and T2,and must have window and trigger method,the window is cut the T1 and T2, the trigger is

Re: modify coGroup GlobalWindows GlobalWindow

2016-09-06 Thread Timo Walther
Hi, will words2 always remain constant? If yes, you don't have to create a stream out of it and coGroup it, but you could simply pass the collection to Map/FlatMap function and do the joining there without the need of a window. Btw. you know that non-keyed global windows do not scale? If I

Re: 回复:Re: fromParallelCollection

2016-09-06 Thread Timo Walther
Hi, you have to implement a class that extends "org.apache.flink.util.SplittableIterator". The runtime will ask this class for multiple "java.util.Iterator"s over your split data. How you split your data and how an iterator looks like depends on your data and implementation. If you need

modify coGroup GlobalWindows GlobalWindow

2016-09-06 Thread rimin515
Hi, the follow code: val text = env.socketTextStream(hostName, port)val words1 = text.map { x => val res = x.split(",") (res.apply(0)->res.apply(1))} val words2 = env.fromElements(("a","w1"),("a","w2"),("c","w3"),("d","w4")) val joinedStream = words1

回复:Re: fromParallelCollection

2016-09-06 Thread rimin515
fromCollection is not parallelization,the data is huge,so i want to use env.fromParallelCollection(data),but the data i do not know how to initialize, - 原始邮件 - 发件人:Maximilian Michels 收件人:"user@flink.apache.org" , rimin...@sina.cn 主题:Re: