Hi Biao, you're right that the BlobServer won't live in the JM in FLIP-6. Instead it will either be part of the RM or the dispatcher component depending on the actual implementation. The requirements for the BlobServer should, however, be the same.
Concerning the question about Flink's distributed cache, I think this is an orthogonal topic. I think that we should be able to piggy-back on the BlobServer once we have refactored it. This should simplify the DC's implementation a bit. Cheers, Till On Thu, Jun 15, 2017 at 9:16 AM, Biao Liu <mmyy1...@gmail.com> wrote: > I have the same concern with Chesnay Schepler. AFIK Flink does not support > DC as well as Mapreduce and Spark. We only support DC in DataSet API. And > DC in flink do not support local files. Is this a good change to refactor > DC too? > > I have another concern, currently BLOB server has some conflicts with > FLIP-6 architecture. We start JM while submitting job instead of starting > it before in FLIP-6. If BLOB server is embedded in JM we can not upload > jars and files before JM started. But the fact is that we need jars > uploaded before starting JM. Correct me is I was wrong. > To solve this problem we can separate submitting job into different stage. > Or we can separate BLOB server as a independent component parallel with RM. > > Maybe we can think more about these in FLIP-19, what do you think? @Nico >
