sounds like a good idea to me. +1 2015-02-17 11:28 GMT+01:00 Stephan Ewen <se...@apache.org>:
> Hi everyone! > > What do you think about making the streaming execution mode of the system > explicit? That means that people start a Flink cluster explicitly in Batch > mode or in Streaming mode. > > The rational behind this idea is that I am not sure how batch and streaming > clusters are really shared in a meaningful way, since streaming programs > basically run forever. There are also further differences: > > - Memory Management: Streaming jobs do not use the managed memory > currently (see [1] and [2]) > > - Are streaming jobs inherently single user? Initially, I would say yes, > because you need to know that you provisioned enough compute power to keep > up with your ingestion rate and that not some other job starts eating > shared resources from you (network / disk) > > - High Availability will probably look a bit different for a streaming > master and a batch master > > Once we figured the co-existence between streaming and batch in the same > cluster out better, we can remove this separation. This does not affect any > user programs, only the "ops" of the cluster. > > Greetings, > Stephan > > > [1] https://issues.apache.org/jira/browse/FLINK-1368 > [2] https://issues.apache.org/jira/browse/FLINK-1323 >
