[ 
https://issues.apache.org/jira/browse/NIFI-16583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18145887#comment-18145887
 ] 

David Handermann commented on NIFI-16583:
-----------------------------------------

Thanks for describing the proposed components and providing the GitHub 
repository for the current implementation.

Although the C++ implementation of MiNiFi currently provides support for OPC 
UA, bringing this into the Java implementation would require additional 
committer knowledge and expected support for future maintenance. Other 
committers may provide more detailed responses, but I'm not sure there is 
sufficient support for something like this right now.

If you don't receive other responses, the next best step is to consider 
publishing the NAR binaries directly through Maven Central. This would make it 
easier to consume and use for current Apache NiFi users, providing the 
opportunity to evaluate the level of interest and potential support at the 
community level.

> Add ConsumeOPCUA processor for OPC UA data-change subscriptions
> ---------------------------------------------------------------
>
>                 Key: NIFI-16583
>                 URL: https://issues.apache.org/jira/browse/NIFI-16583
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>            Reporter: Simon Theodosius
>            Priority: Major
>              Labels: opcua
>
> h3. Motivation
> OPC UA servers expose industrial measurements and their quality and 
> timestamps. A NiFi source processor that subscribes to value changes would 
> allow these measurements to enter a dataflow directly, with configurable 
> sampling and batching.
> I propose adding ConsumeOPCUA using Eclipse Milo and adapting an existing 
> standalone implementation to the current Apache NiFi codebase.
> h3. Proposed scope
> * Subscribe to OPC UA Value data changes over opc.tcp and emit initial values 
> and subsequent changes as batched FlowFiles.
> * Configure tags using namespace-index or namespace-URI NodeIds, with 
> aliases, sampling intervals, server queue sizes and deadbands.
> * Include tag identity, value, status and timestamps in the output.
> * Support anonymous and username/password authentication, explicit security 
> policy and message security mode selection, and client application 
> certificates. Full server certificate validation should be the default.
> * Support connection verification, reconnection and subscription recreation, 
> with primary-node execution in clustered NiFi.
> OPC UA Events/alarms, historical reads, writes and durable recovery are 
> outside the proposed initial scope.
> h3. Delivery and resource limits
> Bound retained notifications by count and serialized bytes. Keep batches 
> reserved and counted against both limits until NiFi reports the commit 
> outcome, and report buffer overflow and server-reported notification loss. 
> These limits do not bound all decoded protocol objects or total JVM heap.
> Shutdown should stop accepting notifications and make a time-limited attempt 
> to drain buffered data. This does not impose a deadline on NiFi repository 
> operations or the overall processor stop.
> Uncommitted notifications remain in memory and are not replicated. Outages, 
> overflow, JVM termination and primary-node failover can cause gaps or 
> duplicates. The processor would not provide end-to-end exactly-once or 
> lossless outage recovery.
> h3. Existing implementation and design questions
> The standalone implementation is available at 
> https://github.com/paf91/nifi-opcua and is currently published under the 
> processor name SubscribeOpcUa. ConsumeOPCUA is the proposed upstream name. 
> Integration into the Apache NiFi source tree and dependency/licensing review 
> remain to be completed.
> I would appreciate maintainer guidance on two initial design choices:
> * Start with JSON/NDJSON output, or expose a RecordSetWriterFactory in the 
> first contribution?
> * Keep OPC UA identity/trust configuration on the processor, or introduce an 
> OPC UA-specific Controller Service?
> h3. Proposed verification
> * Exercise supported authentication/security combinations and certificate 
> failures.
> * Cover namespace resolution, value encoding, reconnect/resubscribe, buffer 
> saturation, downstream backpressure, commit failures and Stop/Start races.
> * Measure memory and CPU use under representative load and repeated 
> connection failures.
> * Verify interoperability with an independent OPC UA server such as open62541 
> and load the resulting NAR in NiFi.
> * Complete the current NiFi contribution checks, dependency/licensing review 
> and processor documentation.
> h3. Related work
> * MINIFICPP-819 - OPC Unified Architecture Support.
> * MINIFICPP-893 - OPC UA Client Implementation.
> * MINIFICPP-2908 - Add processor to fetch OPC events.
> These concern MiNiFi C++. This proposal targets the Java NiFi runtime and 
> Value data-change subscriptions.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to