Hi guys,
+0 (non-binding)
Checked signature and checksum. Built sources without issue. Used toolkit
to convert template. And ran a template to exchange data between a MiNiFi
instance and a NiFi instance. Great !!
(under Windows 10 environment)
Few remarks though (sorry if it has already been mentioned):
1. CRON driven support
It seems to me that CRON driven processors are not supported (following
stack trace when running MiNiFi with a template with a CRON driven
processor):
2016-07-09 15:18:08,492 ERROR [main] org.apache.nifi.minifi.MiNiFi Failure
to launch MiNiFi due to java.lang.NoClassDefFoundError:
org/quartz/CronExpression
java.lang.NoClassDefFoundError: org/quartz/CronExpression
at
org.apache.nifi.controller.StandardProcessorNode.setScheduldingPeriod(StandardProcessorNode.java:446)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.controller.StandardFlowSynchronizer.updateProcessor(StandardFlowSynchronizer.java:721)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:776)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:271)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1287)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:72)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:629)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:479)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at org.apache.nifi.minifi.MiNiFiServer.start(MiNiFiServer.java:75)
~[minifi-framework-core-0.0.1.jar:0.0.1]
at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:134)
~[minifi-runtime-0.0.1.jar:0.0.1]
at org.apache.nifi.minifi.MiNiFi.main(MiNiFi.java:233)
~[minifi-runtime-0.0.1.jar:0.0.1]
Caused by: java.lang.ClassNotFoundException: org.quartz.CronExpression
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
~[na:1.8.0_74]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_74]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
~[na:1.8.0_74]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_74]
... 11 common frames omitted
If it is expected, it'd be nice to have a warning while converting
templates.
2. Relationships conversion
I have a processor connected (for both success and failure relationships)
to a remote process group in my original template (XML). When converting
into a YAML template, only the failure relationship is here and I get the
following error:
2016-07-09 15:24:58,921 ERROR [main] o.a.nifi.groups.StandardProcessGroup
Unable to start AttributesToJSON[id=AttributesToJSON] due to {}
java.lang.IllegalStateException: Processor AttributesToJSON is not in a
valid state due to ['Relationship success' is invalid because Relationship
'success' is not connected to any component and is not auto-terminated]
at
org.apache.nifi.controller.StandardProcessorNode.start(StandardProcessorNode.java:1235)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.controller.scheduling.StandardProcessScheduler.startProcessor(StandardProcessScheduler.java:321)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.groups.StandardProcessGroup.startProcessor(StandardProcessGroup.java:1047)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at
org.apache.nifi.groups.StandardProcessGroup.startProcessing(StandardProcessGroup.java:268)
~[nifi-framework-core-0.6.1.jar:0.6.1]
at org.apache.nifi.minifi.MiNiFiServer.start(MiNiFiServer.java:77)
[minifi-framework-core-0.0.1.jar:0.0.1]
at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:134)
[minifi-runtime-0.0.1.jar:0.0.1]
at org.apache.nifi.minifi.MiNiFi.main(MiNiFi.java:233)
[minifi-runtime-0.0.1.jar:0.0.1]
I had to manually modify the YAML configuration to get things working.
Maybe I missed something here.
In any case, not a big deal from my point of view. Let me know if I should
raised a JIRA.
For interested people, I wrote a short article about what I did to test
MiNiFi (first time I was looking into it):
https://pierrevillard.com/2016/07/09/apache-nifi-minifi-is-almost-out/
Great job!!
Pierre
2016-07-08 23:26 GMT+02:00 Mark Payne <[email protected]>:
> And since i originally replied to the helper thread instead of the VOTE
> thread...
>
> +1 (binding)
>
> Verified signature and checksums. Build completed without issue. Was able
> to use the minifi-toolkit to convert
> a nifi template and run it without any problem. All worked perfectly!
> Great job, guys!
>
>
> > On Jul 8, 2016, at 1:46 PM, Aldrin Piri <[email protected]> wrote:
> >
> > +1, binding
> >
> > Build, contrib, and associated documentation looks good.
> >
> > Hashes and signatures check out.
> >
> > Was able to use the generated assemblies to perform some tests and
> > communicate with a NiFi instance.
> >
> > ---
> >
> > Folks,
> >
> > Thanks for all the feedback. I've created tickets to capture all of the
> > points made. Much appreciated and happy to have some additional eyes on
> > the effort.
> >
> >
> > On Fri, Jul 8, 2016 at 1:27 PM, Andy LoPresto <[email protected]>
> wrote:
> >
> >> My +1 vote from the other day is now binding.
> >>
> >> Andy LoPresto
> >> [email protected]
> >> *[email protected] <[email protected]>*
> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >>
> >> On Jul 8, 2016, at 9:12 AM, Joe Witt <[email protected]> wrote:
> >>
> >> +1 binding.
> >>
> >> Ran through full build verification. Check licensing/notice items and
> >> hoping we can make that leaner and meaner over time as well.
> >>
> >> Was able to use the toolkit to convert a nifi template to a minifi
> >> config and fire that up and immediately see desired results.
> >>
> >> Some feedback items.
> >>
> >> 1) minifi-toolkit awareness
> >> The build helper email references this which makes knowledge of it
> >> available. But I didn't initially notice any references to it in the
> >> system admin guide doc or README entries. Might need to call it out
> >> more or provide a more prescriptive pointer to how to build it, find
> >> it, use it.
> >>
> >> 2) minifi-toolkit execution
> >>
> >> When running config.sh it says
> >> "
> >> Transform Usage:
> >>
> >> java org.apache.nifi.minifi.toolkit.configuration.ConfigMain transform
> >> INPUT_FILE OUTPUT_FILE
> >> "
> >>
> >> But I ended up just running 'bin/config.sh IN.xml OUT.yml' and it
> >> worked great so the java classname bits there are probably superfluous
> >> or only apply to those running without the script.
> >>
> >> 3) WARN in logs. Probably can be pulled in differently to avoid this.
> >> 2016-07-08 09:04:30,300 WARN [main] org.apache.nifi.nar.NarClassLoader
> >> ./work/nar/framework/minifi-framework-nar-0.0.1.nar-unpacked does not
> >> contain META-INF/bundled-dependencies!
> >>
> >> 4) Some of these things about how minifi starts up, is configured, API
> >> items, are probably still subject to change. We should document that
> >> well. A version of 0.0.1 might not be sufficient to communicate that.
> >>
> >> On Fri, Jul 8, 2016 at 7:56 AM, Bryan Rosander <[email protected]
> >
> >> wrote:
> >>
> >> +1 (non-binding)
> >>
> >> Downloaded RC, verified checksums, gpg.
> >>
> >> Built with contrib-check turned on.
> >>
> >> Ran through all templates at
> >>
> >>
> https://github.com/brosander/minifi-testkit/tree/8a8b9791a6b3d6404d6eaa6be48951851d3fa898/templates
> >> (ran config.sh transform on each template to generate yaml, ran MiNiFi
> with
> >> resulting yaml)
> >>
> >> Thanks,
> >> Bryan
> >>
> >> On Fri, Jul 8, 2016 at 10:16 AM, Joe Percivall <
> >> [email protected]> wrote:
> >>
> >> +1 (binding)
> >>
> >> Ran through the helper and verified functionality.
> >>
> >> - - - - - -
> >> Joseph Percivall
> >> linkedin.com/in/Percivall
> >> e: [email protected]
> >>
> >>
> >>
> >>
> >> On Thursday, July 7, 2016 2:13 PM, Joe Percivall
> >> <[email protected]> wrote:
> >> Vote forth-coming but regarding Andy's notes/questions:
> >> I agree with using HTTPS links when possible in both files and the
> couple
> >> README.md changes.
> >>
> >> I don't think there are any plans currently for separate mailing
> lists/IRC
> >> nodes. I am in favor of keeping NiFi and MiNiFi together for now to
> build
> >> visibility, especially since MiNiFi is a sub-project of NiFi. Once
> there is
> >> a steady stream of questions we can revisit it.
> >> If this vote passes, it would be the first release. In order to avoid
> the
> >> issues with the sensitive property you suggest would it need to be done
> for
> >> this release?
> >> Technically Connections are referring to the IDs of both the Processor
> and
> >> Port. The difference is that the Processor's IDs are set as the same as
> the
> >> configured name and for the Ports, the IDs and names are those that are
> set
> >> in the target NiFi instance (you're point to a component in another
> >> system). As you bring up this adds the the limitation that all names
> must
> >> be unique (which in small deployments that MiNiFi targets shouldn't be a
> >> problem) but makes it more user friendly. Enforcing uniqueness was
> added as
> >> part of a contribution from Bryan Rosander as a validation check when
> >> transforming the config.yml and applies to processors, connections, and
> >> remote processing groups.
> >> Joe
> >> - - - - - - Joseph Percivalllinkedin.com/in/Percivalle:
> >> [email protected]
> >>
> >>
> >>
> >> On Thursday, July 7, 2016 12:05 AM, Andy LoPresto <
> >> [email protected]> wrote:
> >>
> >>
> >> Hi,
> >> I verified the GPG signature and all hash signatures. Contrib check ran
> >> fine.
> >> I used one of the example config.yml files in the System Admin Guide and
> >> deployed it to the compiled binary. The tool ran successfully and wrote
> to
> >> a log file on my system. I have a couple notes about the release but
> >> nothing blocking.
> >> +1 (non-binding)
> >> Notes/questions:
> >> README.md:* Could we use HTTPS links when possible?* The note about the
> >> time to run a parallel Maven build says on a moderate dev laptop it
> should
> >> complete in less than 10 minutes. While true, a more accurate statement
> >> would be < 2 minutes. * The mkdir example references
> “example-nifi-deploy”
> >> and “example-minifi-deploy” interchangeably. * Are there separate
> MiNiFi
> >> mailing lists and IRC nodes coming?
> >> System Admin Guide:* Could we use HTTPS links when possible?
> >> config.yml:* Could we please change the default algorithm for protecting
> >> sensitive property values to something stronger than the current
> selection?
> >> I would open a Jira if necessary, but this is one of those things that
> is
> >> really better to do before the first release so users have a
> >> backward-compatible config.yml file moving forward. If we change it in a
> >> subsequent release, we will need to do significant migration
> hand-holding.
> >> My suggestion would be "PBEWITHSHA256AND256BITAES-CBC-BC” which is
> >> significantly stronger, but after trying a few BC options, I continue to
> >> get EncryptionExceptions even though I have the JCE unlimited
> cryptographic
> >> strength jurisdiction policy files installed, so this may be a 0.0.2
> fix.
> >> Is BouncyCastle not enabled by default?* Question about how processors
> and
> >> connectors reference each other — it appears connectors reference source
> >> processors by name but destination processors or ports by ID? Is this
> >> consistent or just demonstrating the flexibility of the system? Also,
> what
> >> enforces processor name uniqueness, as this is not an existing feature
> of
> >> NiFi.
> >> All in all looks like a lot of great work and will expand the user base
> >> and opportunities for NiFi significantly. Thanks.
> >> Andy [email protected]@gmail.comPGP
> >> Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> >>
> >> On Jul 6, 2016, at 7:27 PM, Aldrin Piri <[email protected]> wrote:
> >> Hello Apache NiFi Community,
> >>
> >> I am pleased to be calling this vote for the source release of Apache
> NiFi
> >> - MiNiFi,
> >> minifi-0.0.1.
> >>
> >> The source zip, including signatures, digests, etc. can be found at:
> >> https://repository.apache.org/content/repositories/orgapachenifi-1087
> >>
> >> The Git tag is minifi-0.0.1-RC1
> >>
> >> The Git commit hash is d6ebeda6bce8259f0531e8455965625333315bdb
> >> *
> >>
> >>
> >>
> https://git-wip-us.apache.org/repos/asf?p=nifi-minifi.git;a=commit;h=d6ebeda6bce8259f0531e8455965625333315bdb
> >>
> >> *
> >>
> >>
> >>
> https://github.com/apache/nifi-minifi/commit/d6ebeda6bce8259f0531e8455965625333315bdb
> >>
> >> Checksums of minifi-0.0.1-source-release.zip:
> >> MD5: b63355fb419ca09b26fe12485b3bf4a5
> >> SHA1: 4c9352b7d4d53d139a707dbdcd03df73dbfb847d
> >> SHA256: d52e7f614afe6aee05e6796e429f2813b51f36d2b6d83a4c08a1fb1fdb50b2ff
> >>
> >> Release artifacts are signed with the following key:
> >> https://people.apache.org/keys/committer/aldrin
> >>
> >> KEYS file available here:
> >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> >>
> >> 30 issues were closed/resolved for this release:
> >>
> >>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12335481
> >>
> >> Release note highlights can be found here:
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Version0.0.1
> >>
> >> The vote will be open for 72 hours.
> >> Please download the release candidate and evaluate the necessary items
> >> including checking hashes, signatures, build from source, and test. Then
> >> please vote:
> >>
> >> [ ] +1 Release this package as minifi-0.0.1
> >> [ ] +0 no opinion
> >> [ ] -1 Do not release this package because...
> >>
> >> Thanks!
> >>
> >>
> >>
>
>
