Hi,
I am reading data points from a file and then i have to perform iterations
over it.
When I just check the data points before the iteration as follows,
tuples.flatMap(new CheckData())
and print count inside CheckData() then I get 2500 data points each over 4
partitions, i.e. 10000 datapoints overall.
Now when I do the following :
ConnectedIterativeStreams<Point, Centroid[]> inputsAndMicroCluster =
tuples.iterate()
.withFeedbackType(Centroid[].class);
DataStream<Centroid[]> updatedMicroCluster =
inputsAndMicroCluster
.flatMap(new MyCoFlatmap());
inputsAndMicroCluster.closeWith(updatedMicroCluster.broadcast());
Then, inside the MyCoFlatmap() method in map1, I check the count and its
less than 1/5th of the total number of datapoints.
I don't really know what is happening here exactly, why would the number of
data points reduce like that suddenly?
Thanks and Regards
Biplob Biswas
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Data-point-goes-missing-within-iteration-tp7776.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com.
