Re: [DISCUSS] roadmap for the next 6-12 months
Looks well thought out to me! Thanks to all who put the effort into this! On Fri, Jan 8, 2016 at 6:29 AM Joe Witt wrote: > Hello NiFi Community > > Please review the proposed roadmap and suggest things to add, remove, > update, prioritize differently. > > Over the past year we have had great contributions that improved the > quality of NiFi and the community. As we know there are many JIRAs > related to improvements or new features. However, it can be difficult > for new and existing community members to know when to jump in, how to > help, or when things they are interested in might be able to happen. > So among other things we should provide and maintain a documented > roadmap to help this. > > The following suggested timeline follows pretty strictly against our > previously stated objective of having releases roughly every 6 weeks. > We do not need to stay strict to this but a time bound (as opposed to > feature bound) release cycle does help provide a regular cadence to > follow and helps give contributors a sense of when their efforts might > end up in a release. > > The items below are largely just focused on some of the major muscle > movements needed at a framework level. There will of course be many > many more JIRAs with each releas. > > NiFi 0.5.0 (Feb 5) > > State Management > https://cwiki.apache.org/confluence/display/NIFI/State+Management > Kerberos > > https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication > Interactive Queue > > https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management > Scripting Support > https://issues.apache.org/jira/browse/NIFI-210 > > NiFi 0.6.0 (Mar 18) > > Deterministic Template Export > https://issues.apache.org/jira/browse/NIFI-826 > Schema/Format Editor (JSON to JSON,…?) > Numerous JIRAs related to format/schema conversion. An example > implementation: > > https://github.com/fsauer65/NiFi-Extensions/tree/master/nifi-jsontransform-bundle > > NiFi 0.7.0 (Apr 29) > > NiFi 1.0.0 (Jun 17) > > HA Cluster Management > https://cwiki.apache.org/confluence/display/NIFI/Clustering+Redesign > HA Data > > https://cwiki.apache.org/confluence/display/NIFI/High+Availability+Processing > Multi-Tenant Authorization > > https://cwiki.apache.org/confluence/display/NIFI/Multi-Tentant+Dataflow > Improved Authorization API > > https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication > NiFi's current authority provider gathers data then makes a decision. > We should instead support PDP/PEP model [4]. > Modernized UX > > https://cwiki.apache.org/confluence/display/NIFI/Redesign+User+Interface > Dependent Properties > https://issues.apache.org/jira/browse/NIFI-1121 > > NiFi 1.1.0 (Sep?) > > Template/Extension Registry > > https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry > Variable Registry > https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry > > The timing of these things is pretty important as far as > interdependencies and how long they'll take to develop and validate. > State management in 050 sets up an important way to have cluster-wide > shared state and introduces zookeeper formally into NiFi which > obviously will be important for the distributed systems behaviors > we'll need as we restructure how clustering works and introduce data > replication for HA data processing. The 060 and 070 releases are > intentionally quite light to allow for very focused work on a parallel > branch of 1.x which probably should start right on the heels of 050. > The 1.0 release will introduce some very key features, offer a chance > to remove deprecated classes, ensure all APIs are extremely clearly > documented for stability and audience, allow us to introduce UI/UX > changes, and so on. > > Now, of course this is simply a proposal and these timings may not > work out. But this should serve as a good basis for discussion and > consensus forming. > > As we progress I plan to update the roadmap [1], product requirements > [2], and feature proposals [3] to reflect these things a bit more > clearly than they are now. > > We should also have a discussion on how long we should be committed to > supporting the 0.x line and what that means. We need to document a > commitment for the community. > > [1] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=58851850 > [2] https://cwiki.apache.org/confluence/display/NIFI/Product+requirements > [3] > https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals > [4] https://en.wikipedia.org/wiki/Common_Open_Policy_Service > > Thanks > Joe >
Re: [DISCUSS] roadmap for the next 6-12 months
Joe! Looks amazing! Thanks for pulling this together Hello NiFi Community Please review the proposed roadmap and suggest things to add, remove, update, prioritize differently. Over the past year we have had great contributions that improved the quality of NiFi and the community. As we know there are many JIRAs related to improvements or new features. However, it can be difficult for new and existing community members to know when to jump in, how to help, or when things they are interested in might be able to happen. So among other things we should provide and maintain a documented roadmap to help this. The following suggested timeline follows pretty strictly against our previously stated objective of having releases roughly every 6 weeks. We do not need to stay strict to this but a time bound (as opposed to feature bound) release cycle does help provide a regular cadence to follow and helps give contributors a sense of when their efforts might end up in a release. The items below are largely just focused on some of the major muscle movements needed at a framework level. There will of course be many many more JIRAs with each releas. NiFi 0.5.0 (Feb 5) State Management https://cwiki.apache.org/confluence/display/NIFI/State+Management Kerberos https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication Interactive Queue https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management Scripting Support https://issues.apache.org/jira/browse/NIFI-210 NiFi 0.6.0 (Mar 18) Deterministic Template Export https://issues.apache.org/jira/browse/NIFI-826 Schema/Format Editor (JSON to JSON,…?) Numerous JIRAs related to format/schema conversion. An example implementation: https://github.com/fsauer65/NiFi-Extensions/tree/master/nifi-jsontransform-bundle NiFi 0.7.0 (Apr 29) NiFi 1.0.0 (Jun 17) HA Cluster Management https://cwiki.apache.org/confluence/display/NIFI/Clustering+Redesign HA Data https://cwiki.apache.org/confluence/display/NIFI/High+Availability+Processing Multi-Tenant Authorization https://cwiki.apache.org/confluence/display/NIFI/Multi-Tentant+Dataflow Improved Authorization API https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication NiFi's current authority provider gathers data then makes a decision. We should instead support PDP/PEP model [4]. Modernized UX https://cwiki.apache.org/confluence/display/NIFI/Redesign+User+Interface Dependent Properties https://issues.apache.org/jira/browse/NIFI-1121 NiFi 1.1.0 (Sep?) Template/Extension Registry https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry Variable Registry https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry The timing of these things is pretty important as far as interdependencies and how long they'll take to develop and validate. State management in 050 sets up an important way to have cluster-wide shared state and introduces zookeeper formally into NiFi which obviously will be important for the distributed systems behaviors we'll need as we restructure how clustering works and introduce data replication for HA data processing. The 060 and 070 releases are intentionally quite light to allow for very focused work on a parallel branch of 1.x which probably should start right on the heels of 050. The 1.0 release will introduce some very key features, offer a chance to remove deprecated classes, ensure all APIs are extremely clearly documented for stability and audience, allow us to introduce UI/UX changes, and so on. Now, of course this is simply a proposal and these timings may not work out. But this should serve as a good basis for discussion and consensus forming. As we progress I plan to update the roadmap [1], product requirements [2], and feature proposals [3] to reflect these things a bit more clearly than they are now. We should also have a discussion on how long we should be committed to supporting the 0.x line and what that means. We need to document a commitment for the community. [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=58851850 [2] https://cwiki.apache.org/confluence/display/NIFI/Product+requirements [3] https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals [4] https://en.wikipedia.org/wiki/Common_Open_Policy_Service Thanks Joe
Re: [DISCUSS] roadmap for the next 6-12 months
On Fri, Jan 8, 2016 at 8:29 AM, Joe Witt wrote: > We should also have a discussion on how long we should be committed to > supporting the 0.x line and what that means. We need to document a > commitment for the community. Worth a dedicated thread? Presuming that we're going to use "1.0.0" as the start of a stricter versioning policy, I think the first issue is deciding how long we'll support major versions once that happens. If the expectation is that "pre-1.0" versions involve more latitude as we work out what NiFi needs to better find a fit for the public good, then I think it's reasonable for our continuing support period to be less than a 'normal' major version. But we have to know what that looks like first. -- Sean
Re: [DISCUSS] roadmap for the next 6-12 months
Team, I had a lengthy conversation today with Matt Gilman to talk through specifics of the Authorizer API concept. The ideas are captured here https://cwiki.apache.org/confluence/display/NIFI/Support+Authorizer+API. Definitely would like to hear ideas from people experienced in working with centralized authority providers. This will be an important step to making Multi-tenant authorization possible. Thanks Joe On Fri, Jan 8, 2016 at 11:25 AM, Tony Kurc wrote: > Joe! Looks amazing! Thanks for pulling this together > Hello NiFi Community > > Please review the proposed roadmap and suggest things to add, remove, > update, prioritize differently. > > Over the past year we have had great contributions that improved the > quality of NiFi and the community. As we know there are many JIRAs > related to improvements or new features. However, it can be difficult > for new and existing community members to know when to jump in, how to > help, or when things they are interested in might be able to happen. > So among other things we should provide and maintain a documented > roadmap to help this. > > The following suggested timeline follows pretty strictly against our > previously stated objective of having releases roughly every 6 weeks. > We do not need to stay strict to this but a time bound (as opposed to > feature bound) release cycle does help provide a regular cadence to > follow and helps give contributors a sense of when their efforts might > end up in a release. > > The items below are largely just focused on some of the major muscle > movements needed at a framework level. There will of course be many > many more JIRAs with each releas. > > NiFi 0.5.0 (Feb 5) > > State Management > https://cwiki.apache.org/confluence/display/NIFI/State+Management > Kerberos > > https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication > Interactive Queue > > https://cwiki.apache.org/confluence/display/NIFI/Interactive+Queue+Management > Scripting Support > https://issues.apache.org/jira/browse/NIFI-210 > > NiFi 0.6.0 (Mar 18) > > Deterministic Template Export > https://issues.apache.org/jira/browse/NIFI-826 > Schema/Format Editor (JSON to JSON,…?) > Numerous JIRAs related to format/schema conversion. An example > implementation: > > https://github.com/fsauer65/NiFi-Extensions/tree/master/nifi-jsontransform-bundle > > NiFi 0.7.0 (Apr 29) > > NiFi 1.0.0 (Jun 17) > > HA Cluster Management > https://cwiki.apache.org/confluence/display/NIFI/Clustering+Redesign > HA Data > > https://cwiki.apache.org/confluence/display/NIFI/High+Availability+Processing > Multi-Tenant Authorization > https://cwiki.apache.org/confluence/display/NIFI/Multi-Tentant+Dataflow > Improved Authorization API > > https://cwiki.apache.org/confluence/display/NIFI/Pluggable+Authentication > NiFi's current authority provider gathers data then makes a decision. > We should instead support PDP/PEP model [4]. > Modernized UX > https://cwiki.apache.org/confluence/display/NIFI/Redesign+User+Interface > Dependent Properties > https://issues.apache.org/jira/browse/NIFI-1121 > > NiFi 1.1.0 (Sep?) > > Template/Extension Registry > https://cwiki.apache.org/confluence/display/NIFI/Extension%2C+Template%2C+Dataset+Registry > Variable Registry > https://cwiki.apache.org/confluence/display/NIFI/Variable+Registry > > The timing of these things is pretty important as far as > interdependencies and how long they'll take to develop and validate. > State management in 050 sets up an important way to have cluster-wide > shared state and introduces zookeeper formally into NiFi which > obviously will be important for the distributed systems behaviors > we'll need as we restructure how clustering works and introduce data > replication for HA data processing. The 060 and 070 releases are > intentionally quite light to allow for very focused work on a parallel > branch of 1.x which probably should start right on the heels of 050. > The 1.0 release will introduce some very key features, offer a chance > to remove deprecated classes, ensure all APIs are extremely clearly > documented for stability and audience, allow us to introduce UI/UX > changes, and so on. > > Now, of course this is simply a proposal and these timings may not > work out. But this should serve as a good basis for discussion and > consensus forming. > > As we progress I plan to update the roadmap [1], product requirements > [2], and feature proposals [3] to reflect these things a bit more > clearly than they are now. > > We should also have a discussion on how long we should be committed to > supporting the 0.x line and what that means. We need to document a > commitment for the community. > > [1] > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=58851850 > [2] https://cwiki.apache.org/confluence/display/NIFI/Product+requirements > [3] https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals > [