Re: proper way in nifi to sync status between custom processors

Hi Ben, Excuse me, I'm trying, but probably I don't fully understand what you want to achieve with the flow. It looks weird that WaitBatch is failing with such FlowFile repository error, while other processor such as ReplaceText succeeds. I recommend to test WaitBatch alone first without combinin

Re: proper way in nifi to sync status between custom processors

Thanks Koji, for the ExecuteSqlCommand issue, I was trying to re-execute the sql query if the connection is lost(connection could be unstable), my idea is to only transfer the FlowFile to the success relationship after successfully executing the sql query. You could see the do while loop in the cod

Re: proper way in nifi to sync status between custom processors

Hi Ben, The ExecuteSqlCommand retry logic does not execute the same query multiple times if it succeeds. So, there must be input FlowFiles containing the same query had been passed more than once. It could be the same FlowFile, or different FlowFiles generated by the first processor for some reaso

Re: proper way in nifi to sync status between custom processors

Hi Koji, thanks, the names of the temp tables are created with format "MMddHHmmssSSS-", the first time indicates the time and the second part is a random number with length of 4. So I think it's not possible to have 2 duplicate table names, the only possibly I could think is the flowfile is

Re: proper way in nifi to sync status between custom processors

Hi Koji, sorry about the provenance exception, it was because there's no space left on the machine(filled up with logs) Regards, Ben 2017-12-27 17:11 GMT+08:00 尹文才 : > Hi Koji, thanks, the names of the temp tables are created with format > "MMddHHmmssSSS-", the first time indicates the t

Re: proper way in nifi to sync status between custom processors

I see, thanks. The easiest way to look at provenance events would be by right clicking a processor instance you are interested in, then select 'View data provenance' context menu. This way, NiFi displays provenance events for the selected processor. Koji On Wed, Dec 27, 2017 at 6:17 PM, 尹文才 wrot

Re: proper way in nifi to sync status between custom processors

Hi Koji, I saw it was only showing the 1000 events so I couldn't see the event when the FlowFile was created. Regards, Ben 2017-12-27 17:21 GMT+08:00 Koji Kawamura : > I see, thanks. The easiest way to look at provenance events would be > by right clicking a processor instance you are interested

Re: proper way in nifi to sync status between custom processors

Hi Ben, you can filter events by timestamp as well. https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#searching-for-events On Wed, Dec 27, 2017 at 6:28 PM, 尹文才 wrote: > Hi Koji, I saw it was only showing the 1000 events so I couldn't see the > event when the FlowFile was created. > > Re

DBCP Connection Pooling using multiple OJDBC Drivers

I'm building a custom processor where i need to execute PL/SQL Procedures with the help of DBCP Connection Pooling Controller Service. The custom processor which executes PL/SQL Procedures needs to connect to different Oracle Databases like 11g and Oracle 8i. The Problem i'm facing here is that t

Compile avro to RecordSchema

Is there an API for compiling an avro schema to a RecordSchema object, or is it more involved than that? Thanks, Mike

Re: DBCP Connection Pooling using multiple OJDBC Drivers

Hi Nadeem, I think you can do very similar to Kafka implementation. For different version NIFI have two different processor for Kafka 10 and 11 “PublishKafka_0_10” and “PublishKafka_0_11” Regards, Milan Das Sr. System Architect email: m...@interset.com

Re: DBCP Connection Pooling using multiple OJDBC Drivers

Thanks Milan, I have doubt here. When i execute my custom processor keeping ojdbc14.jar for Oracle 8i in my nifi-lib folder then custom processor works fine, but when i execute processor for Oracle 11 g then its throwing me an error "NoSuchMethodError Oracle.jdbc.OracleConnection.createOracleArray

Re: DBCP Connection Pooling using multiple OJDBC Drivers

Nadeem, You should not be placing any jar file into nifi’s lib directory. Doing so will load the jar into all classloaders in the system. Instead, you should place the jar outside of the lib/ directory and then configure the controller service to point to the jar file, wherever it is placed.

Re: DBCP Connection Pooling using multiple OJDBC Drivers

Thanks Mark that really helps. But I'm facing below issue As you said i loaded drivers externally and it was successful for oracle8i where i executed simple stored procedure and it is working fine but when i try to execute stored procedure which has array type for Oracle 11g then im getting this

Nifi "event driven" scheduling strategy Custom Processor

I am trying to build a custom processor with “Event Driven”. I added   “@EventDriven” annotation. It is showing EventDriven in the scheduling strategy. Question: how to send an event to this processor to intiate the flow? What are my options? Regards, Milan Das Sr. System Architect email:

Provenance events id disorder bug

Hi! In my attempts to try to get to the bottom of the events disorder bug ( https://issues.apache.org/jira/plugins/servlet/mobile#issue/NIFI-4638) I have written a small script that finds the disorders. I hope you might find this useful in trying to recreate the bug, and find out if you are experi

Re: Provenance events id disorder bug

Sorry, the script is attached to this mail. בתאריך יום ד׳, 27 בדצמ׳ 2017, 20:04, מאת נתנאל ביטאן ‏: > Hi! > > In my attempts to try to get to the bottom of the events disorder bug ( > https://issues.apache.org/jira/plugins/servlet/mobile#issue/NIFI-4638) I > have written a small script that finds

Re: Provenance events id disorder bug

hello it is best to attach the script to a jira. thanks On Dec 27, 2017 1:52 PM, "נתנאל ביטאן" wrote: > Sorry, the script is attached to this mail. > > בתאריך יום ד׳, 27 בדצמ׳ 2017, 20:04, מאת נתנאל ביטאן ‏< > bitaneta...@gmail.com>: > >> Hi! >> >> In my attempts to try to get to the bottom of

Re: Nifi "event driven" scheduling strategy Custom Processor

Hello! An "event" in the context of the NiFi @EventDriven annotation is the arrival of a flow file in the queue that is an input to a processor. The only purpose of @EventDriven is to save system resources when compared to the default Timer Driven strategy, in which a processor polls its input qu

Re: Provenance events id disorder bug

Thanks for your quick comment. I've attached the script to this jira: https://issues.apache.org/jira/plugins/servlet/mobile#issue/NIFI-4638 And uploaded it as a gist in this address: https://gist.github.com/bitanetanel/808cb3a0204aad5830f31d6d7ff89299 ‫בתאריך יום ד׳, 27 בדצמ׳ 2017 ב-21:06 מאת ‪Joe

Re: [Non-DoD Source] Re: Moving from version 1.1.2 to 1.4.0

Steven, Joe and Bryan's suggestions are best, but there is another way to try to resolve your issue, It comes with a large "Beware of class loaders" sign, but it can work. Your custom nar can be built so that it depends on the classes found in nifi-standard-nar, by using a nar dependency. A nar

Re: Compile avro to RecordSchema

Hi Mike, You might already have found it, but AvroTypeUtil.createSchema is probably what you are looking for. https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java#L341 Thanks,

Re: DBCP Connection Pooling using multiple OJDBC Drivers

Hi Nadeem, Did you try specifying a external directory instead of an exact jar location, then put multiple versions of Jar there? This way DBCPConnectionPool can utilize multiple jars. This would provide the similar effect with putting one in NiFi lib dir. If that doesn't work, an alternative app

Re: [DISCUSS] First Release of NiFi Registry

Looks like the work for the initial docs has wrapped up and we are in a good place to kick out an RC. I'll start pulling things together and should be able to get something out tomorrow. Pierre, I created a JIRA [1] to capture an improvement for the scenario you described with the nested process