Sincerely i dont understand how it is developed this project. You are essentially developing a streaming management permits complex internal states, permits to split streams in parallel way , to merge and reduce streams in statefull way. It is great idea. But this is a generic concept. Does this abstract concept need hadoop for exiting ?No. It is a generic and abstract solution for many problems. Does this concept need to know the client-server concept for working and need to be part of a client -server concept for being what is ?No
You developed flink project embedding strict dependencies with hadoop, with yarn in a monolitic project that pratically blocks anyone to use it in a different way from how you embedded inside your solution. You merged a abstract algorithmic idea with many other things not strictly related. In addition you developed it as server solution.You merged a generic algorithm with a concrete application of this algorithm. This method to develop blocks anyone want use your abstract concept for doing something over. The world evolves when a tecnology is easy to reuse for creating other more complex solution over, when the people neednt to go inside what it is invented or delivered for a new idea ,because they can compose your idea with others. This composition process permits to evolves the tecnologies. In short , the right way is to develop short libraries containing the strict logic necesssary for implementing that idea. And then you can implement you solution/application based on it. Think for a moment to another big solution how is lucene. It is developed as a indipendent library. In a decades , tons of open sources and commercial solutions were developed over lucene. The lucene team didnt developed it like a server. Anyway then they developped elasticsearch server. If they would have developed directly elastic search do you think the open source history was the same? Your abstract idea is very good and can be aplied in many different ways . Do not block the future.To rewrite all for a new application based on the same idea requires a effort and money few people /companies have. Sincerelly a so grave error for a open source foundation like Apache foundation is incomprehensible. I hope it could be solved in future for the good of open source future. All the best for the open source world bye
