Re: [DISCUSS] The future of (Mi)NiFi Command and Control (C2)

2021-12-02 Thread Marton Szasz
Hi, I like the idea of using the same protocol to control NiFi, MiNiFi Java and MiNiFi C++. I see no issues with supporting VersionedFlowSnapshot as a config format for MiNiFi C++ as well, but I only checked the schema so far. The C++ agent needs to get a new config parser written from scratch to

Re: [DISCUSS] The future of (Mi)NiFi Command and Control (C2)

2021-11-30 Thread Kevin Doran
Hi Matt, Thanks for kicking of this thread. The discussion here puts into words some things that have been on my mind for a while, and its really exciting to see a revival of these conversations. Overall, I support the proposal. Here are some specific thoughts, in no particular order. One of

Re: [DISCUSS] The future of (Mi)NiFi Command and Control (C2)

2021-11-22 Thread Matt Burgess
I'm the same, historically I've viewed and implemented C2 as process control as well. These days with sophisticated containerization and deployment tools, I'm thinking we leave the process orchestration to them and concentrate on controlling what the processes are doing. All, any other thoughts, q

Re: [DISCUSS] The future of (Mi)NiFi Command and Control (C2)

2021-11-20 Thread Marc Parisi
" This would align it more to the way NiFi works as well, allowing C2 to be used for flow control and metadata exchange rather than process control." This particularly caught my eye and is something that makes sense from an evolutionary perspective. I've previously viewed command and control as a

Re: [DISCUSS] The future of (Mi)NiFi Command and Control (C2)

2021-11-18 Thread Matt Burgess
Marc, That's a great point and reminds me that I missed two whole sections of the proposal: 1) The refactor of minifi-bootstrap, and 2) the standardization of serialized flow control to VersionedFlowSnapshot (i.e. JSON not XML or YAML). 1) Currently it's the bootstrap process that reaches out to

Re: [DISCUSS] The future of (Mi)NiFi Command and Control (C2)

2021-11-18 Thread Marc Parisi
Hi Matt, I think this is great. I like the MVP set of features. Considering use cases and potential maintenance needs, are there thoughts on including RESTART in the MVP for cases where agent UPDATES may be blocked for whatever reason ( I/O, agent problems, etc )? Does the risk of the RESTART ve

[DISCUSS] The future of (Mi)NiFi Command and Control (C2)

2021-11-18 Thread Matt Burgess
Hello all, I'd like to start a discussion about the current and future Command and Control (C2) capabilities in terms of the NiFi/MiNiFi Java codebase. The current implementation[1] is IMO not a pure reference implementation of the C2 spec [2]. It has fairly limited capability and uses the PullHtt