Thanks Matt, Yes, I am subscribed to the development list.
I will review your comments. At the moment I am adding the schemas as additional properties to the ScriptedReader. I will try what you mention to access SchemaRegistry and send you some comments. On Wed, Apr 8, 2020 at 10:30 PM Matt Burgess <mattyb...@apache.org> wrote: > As a follow up, while implementing [1] I ran into [2] which is a > larger issue but affects the RecordSetWriters more than Readers as > they add more properties that are expected to come (in our case) from > the script. That makes things a bit more complicated, but I am still > looking into it. If you are coding a ScriptedReader you might be ok > with the guidance I gave earlier, but I have my doubts that trying to > use a ScriptedRecordSetWriter with a SchemaRegistry will work right > now. I'll post any findings in either/both [1] and [2], and as always > I welcome all comments, questions, and suggestions :) > > Regards, > Matt > > [1] https://issues.apache.org/jira/browse/NIFI-7343 > [2] https://issues.apache.org/jira/browse/NIFI-5115 > > On Wed, Apr 8, 2020 at 8:19 PM Matt Burgess <mattyb...@apache.org> wrote: > > > > Jairo, > > We should probably move this to the dev list since we're getting into > > the NiFi API, but I wasn't sure you were subscribed, if you are let me > > know and we can move it over there. > > > > StandardConfigurationContext is in framework-core so you should not > > count on being able to use that class per se. Instead you should add a > > property like the one in SchemaAccessUtils: > > > > public static final PropertyDescriptor SCHEMA_REGISTRY = new > > PropertyDescriptor.Builder() > > .name("schema-registry") > > .displayName("Schema Registry") > > .description("Specifies the Controller Service to use for the > > Schema Registry") > > .identifiesControllerService(SchemaRegistry.class) > > .required(false) > > .build(); > > > > Then from your code you can add that property in your overridden > > getSupportedPropertyDescriptors() method, and when you need it you can > > get the property from the context, but don't refer to it as a > > StandardConfigurationContext, there's an interface that has the same > > methods and is in the API that you have access to already: > > > > final SchemaRegistry schemaRegistry = > > > context.getProperty(SCHEMA_REGISTRY).asControllerService(SchemaRegistry.class) > > > > Note that we're not getting the registry directly from the context, > > rather we're getting the property value from the context and then > > finding the corresponding SchemaRegistry, I believe that's done "under > > the hood" by the ControllerServiceLookup, the > > ".asControllerService(SchemaRegistry.class)" is the key part there. > > > > From there you can use the SchemaRegistry interface methods to get at > > the other things, for example: > > > > descriptor = getPropertyDescriptor(SCHEMA_ACCESS_STRATEGY.getName()); > > schemaAccess = context.getProperty(descriptor).getValue(); > > schemaAccessStrategy = getSchemaAccessStrategy(schemaAccess, > > schemaRegistry, context); > > > > Note that last method getSchemaAccessStrategy() is from > > SchemaAccessUtils which you don't currently have access to, so you > > might want to just copy all that over to your script. If you follow > > the calls and code from SchemaRegistryService, you should be able to > > bring any/all of that into your ScriptedReader so it can do the same > > kind of processing. > > > > Regards, > > Matt > > > > On Wed, Apr 8, 2020 at 7:03 PM Jairo Henao <jairohenaoro...@gmail.com> > wrote: > > > > > > Thanks Matt, > > > > > > I have made some changes using AvroReader as an example, but I am > still able to access other ControllerServices. > > > > > > From the RecordReaderFactory I have access to a context that is of > type org.apache.nifi.controller.service.StandardConfigurationContext but > with this I don't know how to access the AvroSchemaRegistry. > > > > > > Do you know how I can access the ControllerServiceLookup? > > > > > > > > > > > > > > > > > > > > > On Wed, Apr 8, 2020 at 3:19 PM Matt Burgess <mattyb...@apache.org> > wrote: > > >> > > >> Jairo, > > >> > > >> It is possible for a ScriptedReader to use a SchemaRegistry, although > > >> it isn't currently as easy as it could be. When the Scripted > > >> Reader/Writer components were added [1], much of the reusable code for > > >> schema registry processing was in the service implementation NAR, so > > >> the aforementioned Jira explains that using the SchemaRegistry > > >> interface is certainly possible in ScriptedReader but fairly > > >> cumbersome. Since then the SchemaRegistryService and associated util > > >> classes were moved to a nifi-avro-record-utils [2] so > > >> extensions/components can leverage schema registries. Unfortunately > > >> nifi-avro-record-utils is not currently included in the scripting NAR, > > >> but I have written up [3] to add this. > > >> > > >> In the meantime, take a look at SchemaRegistryService, > > >> SchemaAccessUtils, and an implementation class such as AvroReader, you > > >> could copy and paste all the necessary code to get your ScriptedReader > > >> to interact with a schema registry. > > >> > > >> Regards, > > >> Matt > > >> > > >> [1] https://issues.apache.org/jira/browse/NIFI-3734 > > >> [2] https://issues.apache.org/jira/browse/NIFI-5123 > > >> [3] https://issues.apache.org/jira/browse/NIFI-7343 > > >> > > >> On Wed, Apr 8, 2020 at 2:01 PM Jairo Henao <jairohenaoro...@gmail.com> > wrote: > > >> > > > >> > Hi all, > > >> > > > >> > Is there a way from a ScriptedReader (Controller Service) to read a > schema that is stored in an AvroSchemaRegistry? > > >> > > > >> > I am using Groovy as a language. > > >> > > > >> > Additionally, how can I read additional properties that I have > added to my ControllerService from Groovy code. > > >> > > > >> > Thanks in advance > > >> > > > >> > > > >> > -- > > >> > Saludos > > >> > > > >> > Jairo Henao > > > > > > > > > > > > -- > > > Saludos > > > > > > Jairo Henao > > > > > > Chat Skype: jairo.henao.05 > > > > -- Saludos Jairo Henao *Chat Skype: jairo.henao.05*
