Hi Vishnu
/val env = StreamExecutionEnvironment.getExecutionEnvironment
val jobID = env.getStreamGraph.getJobGraph.getJobID/
As I am using the jobId of current running job. It should exist.
Thanks and regards
Pushpendra
--
View this message in context:
Hi,
I am not broadcasting the data but the model, i.e. the weight vector
contained in the "State".
You are right, it would be better for the implementation with the while
loop to have the data on HDFS. But that's exactly the point of my
question: Why are the Flink Iterations not faster if you
think your anwser.
but i can not get your ideal."If all elements of "words2" have been processed,
the right side of your coGroup will always be empty no matter what is incoming
in your socketTextStream.",the mean i can not get.
the following is the ideal from me(it maybe error):
the
my data from a Hbase table ,it is like a List[rowkey,Map[String,String]],
class MySplittableIterator extends SplittableIterator[String]{
// Members declared in java.util.Iterator
def hasNext(): Boolean = {
}
def next(): Nothing = {
}
// Members
I think you have to rethink your approach. In your example "words2" is a
stream but only with a fixed set of elements. If all elements of
"words2" have been processed, the right side of your coGroup will always
be empty no matter what is incoming in your socketTextStream.
It is not read in
Hi,
this looks like a bug. I created an issue for it
(https://issues.apache.org/jira/browse/FLINK-4581). Could you also send
us the pom.xml you are using for your project?
Timo
Am 06/09/16 um 13:47 schrieb jiecxy:
Hi all,
I want to write a program, a thread read the real-time message
i try read data into a list or List[Map] to store the T2,but i think if use
list or List[Map],there is not parallelization,so i want to use coGroup.
other hand,the coGroup function is join the T1 and T2,and must have window and
trigger method,the window is cut the T1 and T2,
the trigger is
Hi,
will words2 always remain constant? If yes, you don't have to create a
stream out of it and coGroup it, but you could simply pass the
collection to Map/FlatMap function and do the joining there without the
need of a window. Btw. you know that non-keyed global windows do not scale?
If I
Hi,
you have to implement a class that extends
"org.apache.flink.util.SplittableIterator". The runtime will ask this
class for multiple "java.util.Iterator"s over your split data. How you
split your data and how an iterator looks like depends on your data and
implementation.
If you need
Hi, the follow code:
val text = env.socketTextStream(hostName, port)val words1 = text.map {
x => val res = x.split(",") (res.apply(0)->res.apply(1))}
val words2 = env.fromElements(("a","w1"),("a","w2"),("c","w3"),("d","w4"))
val joinedStream = words1
fromCollection is not parallelization,the data is huge,so i want to use
env.fromParallelCollection(data),but the data i do not know how to initialize,
- 原始邮件 -
发件人:Maximilian Michels
收件人:"user@flink.apache.org" , rimin...@sina.cn
主题:Re:
11 matches
Mail list logo