Hi Greg, I don't really know anything about OpenTelemetry, but from the perspective of integrating something into the framework, some things to consider...
Is there some way to piggy-back on provenance and use a ReportingTask to process provenance events and report something to OpenTelemetry? If something new does need to be added, it should probably be an extension point where there is an interface in the framework-api and different implementations can be plugged in. Ideally the framework itself wouldn't have any knowledge of OpenTelemetry specifically, it would only be reporting some information, which could then be used in some way by the OpenTelemetry implementation. How does NiFi actually communicate with OpenTelemetry? Are you expecting to send data to OpenTelemetry in this new method you are suggesting? That would likely have a significant impact on the performance of the flow. Thanks, Bryan On Thu, Jul 28, 2022 at 3:17 PM glma...@uwe.nsa.gov <glma...@uwe.nsa.gov> wrote: > > Nifi Devs, > > My team and I are looking for guidance on how we can extend Apache Nifi's > capabilities. Specifically we're looking to include distributed tracing. > We'll approach this effort as if we're the tracing experts and simply seeking > implementation guidance. Our developers have good exposure to working with > Nifi and creating custom processors. We plan to fork the project to begin > this effort but want to make sure we approach this with the best possible > direction for community adoption. > > Our initial thoughts on this approach would be to piggyback on how Provenance > was implemented. We essentially want to include a subroutine or method that > gets implicitly invoked upon a processors 'onTrigger' method. From there we > would analyze the FlowFiles attributes to check for the existence of > 'traceId' and/or propagate one if found. > > We can expound upon all of these tracing/observability details if that helps > by any means. We're able to provide more detailed scope of this task as well > but for now we just want to get feed back for our overall goal and proposed > approach. > > Thanks, > Greg Marshall
