Sounds. Great Let me know if you need some help
Best regards Craig > On 31 Jul 2019, at 17:31, Arpad Boda <ab...@cloudera.com.invalid> wrote: > > Craig, > > OPC ( https://issues.apache.org/jira/browse/MINIFICPP-819 ) and Modbus ( > https://issues.apache.org/jira/browse/MINIFICPP-897 ) are on the way for > MiNiFi c++, hopefully both will be part of next release (0.7.0). > It's gonna be legen... wait for it! :) > > Regards, > Arpad > >> On Wed, Jul 31, 2019 at 2:30 AM Craig Knell <craig.kn...@gmail.com> wrote: >> >> Hi Folks >> >> That's our use case now. All our Models are run in python. >> Currently we send events to the ML via http, although this is not optimal >> >> Our use case is edge ML where we want a light weight wrapper for >> Python code base. >> Jython however does not work with the code base >> I'm think of changing the interface to some thing like REDIS for pub/sub >> Id also like this to be a push deployment via minifi >> >> Also support for sensors via protocols via Modbus and OPC would be great >> >> Craig >> >>> On Wed, Jul 31, 2019 at 1:43 AM Joe Witt <joe.w...@gmail.com> wrote: >>> >>> Definitely something that I think would really help the community. It >>> might make sense to frame/structure these APIs such that an internal >> option >>> could be available to reduce dependencies and get up and running but that >>> also just as easily a remote implementation where the engine lives and is >>> managed externally could also be supported. >>> >>> Thanks >>> >>> >>> On Tue, Jul 30, 2019 at 1:40 PM Andy LoPresto <alopre...@apache.org> >> wrote: >>> >>>> Yolanda, >>>> >>>> I think this sounds like a great idea and will be very useful to >>>> admins/users, as well as enabling some interesting next-level >> functionality >>>> and insight generation. Thanks for putting this out there. >>>> >>>> Andy LoPresto >>>> alopre...@apache.org >>>> alopresto.apa...@gmail.com >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>> >>>>> On Jul 30, 2019, at 5:55 AM, Yolanda Davis < >> yolanda.m.da...@gmail.com> >>>> wrote: >>>>> >>>>> Hello Everyone, >>>>> >>>>> I wanted to reach out to the community to discuss potentially >> enhancing >>>>> NiFi to include predictive analytics that can help users assess and >>>> predict >>>>> NiFi behavior and performance. Currently NiFi has lots of metrics >>>> available >>>>> for areas including jvm and flow component usage (via component >> status) >>>> as >>>>> well as provenance data which NiFi makes available either through >> the UI >>>> or >>>>> reporting tasks (for consumption by other systems). Past discussions >> in >>>> the >>>>> community cite users shipping this data to applications such as >>>> Prometheus, >>>>> ELK stacks, or Ambari metrics for further analysis in order to >>>>> capture/review performance issues, detect anomalies, and send alerts >> or >>>>> notifications. These systems are efficient in capturing and helping >> to >>>>> analyze these metrics however it requires customization work and >>>> knowledge >>>>> of NiFi operations to provide meaningful analytics within a flow >> context. >>>>> >>>>> In speaking with Matt Burgess and Andy Christianson on this topic we >> feel >>>>> that there is an opportunity to introduce an analytics framework that >>>> could >>>>> provide users reasonable predictions on key performance indicators >> for >>>>> flows, such as back pressure and flow rate, to help administrators >>>> improve >>>>> operational management of NiFi clusters. This framework could offer >>>>> several key features: >>>>> >>>>> - Provide a flexible internal analytics engine and model api which >>>>> supports the addition of or enhancement to onboard models >>>>> - Support integration of remote or cloud based ML models >>>>> - Support both traditional and online (incremental) learning >> methods >>>>> - Provide support for model caching (perhaps later inclusion into >> a >>>>> model repository or registry) >>>>> - UI enhancements to display prediction information either in >> existing >>>>> summary data, new data visualizations, or directly within the >>>> flow/canvas >>>>> (where applicable) >>>>> >>>>> For an initial target we thought that back pressure prediction would >> be a >>>>> good starting point for this initiative, given that back pressure >>>> detection >>>>> is a key indicator of flow performance and many of the metrics >> currently >>>>> available would provide enough data points to create a reasonable >>>>> performing model. We have some ideas on how this could be achieved >>>> however >>>>> we wanted to discuss this more with the community to get thoughts >> about >>>>> tackling this work, especially if there are specific use cases or >> other >>>>> factors that should be considered. >>>>> >>>>> Looking forward to everyone's thoughts and input. >>>>> >>>>> Thanks, >>>>> >>>>> -yolanda >>>>> >>>>> -- >>>>> yolanda.m.da...@gmail.com >>>>> @YolandaMDavis >>>> >>>> >> >> >> >> -- >> Regards >> >> Craig Knell >> Mobile: +61 402 128 615 >> Skype: craigknell >>
