Re: CaptureChangeMySQL - throwing Binlog connector communications failure

Hi Anand, I'm not sure what caused the error, but I believe the error is MySQL error 1236. The error can happen with different reasons, you may find this article informative. https://www.percona.com/blog/2014/10/08/mysql-replication-got-fatal-error-1236-causes-and-cures/ Also, there is a JIRA

Re: Anonymization of CSV data with NiFi?

Hi sgreszcz, CSV is structured format, so you can create a schema, and then move into a record-based world of flow files. 1. Scripted Record Writer - implement in script whatever anonymization you like. 2. If you need to replace the value with static or random value, you can also use UpdateRecord

Re: Anonymization of CSV data with NiFi?

Hello You'd have to write custom code for the anonymization logic specifically. But it would be a perfect case for the Record oriented processors. You could have a pretty simple "AnonymizeRecord" processor for example. Thanks On Mon, Nov 12, 2018 at 5:31 PM wrote: > > Hi there, > > I have a

Anonymization of CSV data with NiFi?

Hi there, I have a use case where i need to read incoming CSV files and randomise / anonymise data in certain columns and re-save the file. Would something like this be possible, or would I need more custom code or to use something like airflow to handle the anonymisation before feeding to

Re: Nifi 1.8.0 global menu (hamburger)

Thanks Andy clearing the cache has done the trick. I feel a little embarrassed for not giving it a go the inconsistent behaviour across instances threw me. Thanks again, Dwane From: Dwane Hall Sent: Tuesday, 13 November 2018 6:27:25 AM To: Andy LoPresto;

EnforceOrder processor

I was really confused how EnforceOrder processor works and NiFi documentation made it even more confusing. After some time looking for an explanation, I found this gist which I think was created by a developer who created this

Re: Nifi 1.8.0 global menu (hamburger)

Thanks Andy. I'll give that a go today. Much appreciated as always, Dwane From: Andy LoPresto Sent: Tuesday, 13 November 2018 12:01 AM To: users@nifi.apache.org Subject: Re: Nifi 1.8.0 global menu (hamburger) Dwane, I can dig into this deeper with you

Re: NiFi 1.8 and stuck flowfile in Load Balanced enabled queue

I ended up disconnecting the Node that had the flowfile in the queue and restarted it, and then rejoinednthe cluster. That took care of it, I'll continue to monitor it. I also added in the expire flow files after X minutes. Regards Dano On Mon, Nov 12, 2018, 8:37 AM Same Issue here! Had to

Re: Easiest way to list all NiFi processors in a release.

Hi Vitaly, Yes there is one [1]: GET on /flow/processor-types Easiest way to find out is to use the developer tool of your browser. Anything viewable/doable in the NiFi UI is a doable through REST API endpoints. [1] https://nifi.apache.org/docs/nifi-docs/rest-api/ Thanks, Pierre Le lun. 12

Re: Ingesting golden gate messages to Hbase using Nifi

Enforcing order is really tricky with Kafka. The only way to enforce order is to reduce the # of nodes processing. You can have one NiFi master node read from Kafka and have it distribute the workload to other NiFi nodes and force ordering. Or you may want to batch them up into say 10-15

Easiest way to list all NiFi processors in a release.

Is there a REST API to list all NiFi processors in a current NiFi release? Thanks. STATEMENT OF CONFIDENTIALITY The information contained in this email message and any attachments may be confidential and legally privileged and is intended for the use of the addressee(s) only. If you are not an

Re: Ingesting golden gate messages to Hbase using Nifi

Faisal, BTW I stumbled upon this doc, that explains how HBase GoldenGate handler works in a similar scenario you've described: https://docs.oracle.com/goldengate/bd123210/gg-bd/GADBD/using-hbase-handler.htm#GADBD-GUID-1A9BA580-628B-48BD-9DC0-C3DF9722E0FB They provide an option to generate

Re: Issue with GeoEnrichIP in NiFi 1.8.0

Philippe, Thanks for reporting this. I have been able to replicate the issue. I created [1] to address the issue and just posted a Pull Request. Thanks -Mark [1] https://issues.apache.org/jira/browse/NIFI-5814 On Nov 12, 2018, at 5:36 AM, PEETERS Philippe mailto:philippe.peet...@etnic.be>>

Re: Nulls in input data throwing exceptions when using QueryRecord

Hi Mark, Thank you for your quick response to the issue and PR fixing the null issue with a single projection! Re #1 - I've attached a template exported from NiFi 1.8.0 that demonstrates the issue. Re #4 - I see, yes I'm not sure the correct behaviour here myself. I think if you're working with

Re: NiFi 1.8 and stuck flowfile in Load Balanced enabled queue

Hey Dan, Have looked through the logs to see if there are any WARN or ERROR's indicating what's going on? Thanks -Mark On Nov 12, 2018, at 9:06 AM, dan young mailto:danoyo...@gmail.com>> wrote: Hello, We have two processor groups connected via the new Load Balancing/Round Robin queue.

Re: NiFi 1.8 and stuck flowfile in Load Balanced enabled queue

Same Issue here! Had to reboot the whole cluster to fix that issue. The files stuck a few seconds/minutes in the queue until they get processed. In my case I assume that it was caused by a one-by-one reboot of the cluster nodes, means we normally reboot only one of 8 nodes and wait a few

NiFi 1.8 and stuck flowfile in Load Balanced enabled queue

Hello, We have two processor groups connected via the new Load Balancing/Round Robin queue. It seems that a flowfile is "stuck" in this queue. I've been watching it for some time now. Is there any way to trouble shoot what is stuck in the queue and why? or maybe remove it? I've tried to

CaptureChangeMySQL - throwing Binlog connector communications failure

Thanks Koji for the resolution of previous issue. But I see below error again and again once in a day. There is nothing else in the logs apart from below error. Can you please tell what could be the problem here? Is it from server side or from processor or any network issue? We are not able

Re: Nifi 1.8.0 global menu (hamburger)

Dwane, I can dig into this deeper with you tomorrow, but the simplest preliminary fix is to clear your browser cache. Sometimes JS changes get stuck and can cause these issues. Cheers. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4

Nifi 1.8.0 global menu (hamburger)

Good evening NiFi users, I've recently updated my NiFi instance to 1.8.0 and have been experiencing issues with the global menu (hamburger) in a clustered environment (2 node). The menu works smoothly on a single node instance but when clustered it only appears possible to access the global

Issue with GeoEnrichIP in NiFi 1.8.0

Hello, I've installed Nifi 1.8.0 and import the processes developed and running with nifi 1.7.1. It looks like the GeoEnrichIP doesn't work correctly anymore in 1.8.0. To be sure I've done a clean install of 1.8.0 and defined a simple process: GenerateFlowFile -> GeoEnrichIP -> LogAttribute

Re: nifi issue with sftp server

Hi all. The error was caused neither from NIFI or sftp server but on the freeipa client side running on the sftp server. thanks all On Mon, Nov 5, 2018 at 9:11 AM Alfredo De Luca wrote: > Hi Joe.. have you had any thought about this? I don't know what else to do. > Cheers > > On Tue, Oct 30,