What is the difference between mini-batch vs real time streaming in practice
(not theory)? In theory, I understand mini batch is something that batches in
the given time frame whereas real time streaming is more like do something as
the data arrives but my biggest question is why not have mini batch with epsilon
time frame (say one millisecond) or I would like to understand reason why one
would be an effective solution than other?I recently came across one example
where mini-batch (Apache Spark) is used for Fraud detection and real time
streaming (Apache Flink) used for Fraud Prevention. Someone also commented
saying mini-batches would not be an effective solution for fraud prevention
(since the goal is to prevent the transaction from occurring as it happened) Now
I wonder why this wouldn't be so effective with mini batch (Spark) ? Why is it
not effective to run mini-batch with 1 millisecond latency? Batching is a
technique used everywhere including the OS and the Kernel TCP/IP stack where the
data to the disk or network are indeed buffered so what is the convincing factor
here to say one is more effective than other?Thanks,kant
