All,

As you know, MiNiFi Java is a separate project and codebase than NiFi,
it's my understanding that this was done because MiNiFi Java was a
fledgling effort while NiFi was an established project, and trying to
do rapid development and releases on MiNiFi would cause some
interference with NiFi features, releases, etc.

Since then, MiNiFi Java has advanced in its own right, but lately
development and release cadence has slowed, and AFAICT much of the
effort is to bring NiFi capabilities into MiNiFi, exposing previously
ignored NiFi properties in MiNiFi's config.yml, etc.

I've begun work on getting MiNiFi Java back into the NiFi codebase,
under its own area with its own assembly, bootstrap, etc. This is
being tracked in Jira under MINIFI-422 [1].

The idea is to be able to build a MiNiFi assembly from NiFi and MiNiFi
components, thereby giving MiNiFi capabilities from NiFi without
having to port them to a separate project. Kerberos support is a
shining example of this, as are other features available via NiFi's
various Context objects (where MiNiFi currently has them stubbed out).

The first step was refactoring some NiFi NAR structure in order to
allow for a "headless" NiFi [2][3], the idea being that we could reuse
some existing framework components/NARs without being tied to the UI,
associated webapps, and Jetty itself.

I will create separate [DISCUSS] threads for each of the following,
but here are some things I'm looking at going forward:

1) Bring MiNiFi bootstrap and external command-and-control (C2) code
over to a MiNiFi area in the NiFi project structure. Also create a
MiNiFi assembly in that area for the purposes of generating release
artifacts when ready.

2) Bring over and update the C2 Server reference implementation, for
testing and to give some way to externally update config/flows in
MiNiFi. This may mean we don't need the change ingestors anymore, but
I will send a separate DISCUSS thread to see who's using them, if
they're important to keep, etc.

3) Discuss whether to bring over the config.yml mechanism for
defining/updating configuration and flow. If we eventually want to
share more between MiNiFi and NiFi, should we return to using
nifi.properties and flow.xml.gz rather than maintaining (and updating)
the code to expose nifi properties as human-readable YAML strings, to
include all the schemas and converters? This too will be a separate
[DISCUSS] thread.

4) Once all is determined and implemented, look at ways to share any
components between NiFi and MiNiFi. This might include refactoring the
C2 stuff into module(s) for use by NiFi [4], adding/enabling profiles
and/or assemblies to build various products (headless NiFi with C2,
MiNiFi with Elasticsearch NARs, e.g.)

I plan on starting on 1 soon, but am glad to discuss any of these
things or any other topics you'd like to bring up (for 2 and 3 let's
have the discussion in the forthcoming DISCUSS threads)

Regards,
Matt

[1] https://issues.apache.org/jira/browse/MINIFI-422
[2] https://issues.apache.org/jira/browse/NIFI-7592
[3] https://issues.apache.org/jira/browse/MINIFI-485
[4] https://issues.apache.org/jira/browse/NIFI-6813

Reply via email to