INT96 Error while converting Parquet File

2021-03-25 Thread Bimal Mehta
Hi, I am trying to convert a Parquet file into csv using ConvertRecord (1.13.0) processor. I have created a controller service of type ParquetReader 1.13.0 I have set the property Avro Read Compatibility to false as I understand that this error has its root in Avro dependency. I also tried it by

RE: [EXTERNAL] Re: NiFi Registry SSL question

2021-03-25 Thread Rosso, Roland
Thank you Bryan, I've tried all combinations I could think off. I'll resign all the certs with the same key for nifi and registry and try this again. Thanks, Roland Rosso AdventHealth Big Data Administrator | Corporate Analytics O: 407-805-8532 -Original Message- From: Bryan Bende

RE: RecordPath...With INNER records

2021-03-25 Thread Greene (US), Geoffrey N
Jira Ticket filed #8366 From: Mark Payne [mailto:marka...@hotmail.com] Sent: Thursday, March 25, 2021 9:16 AM To: users@nifi.apache.org Subject: [EXTERNAL] Re: RecordPath...With INNER records EXT email: be mindful of links/attachments. Geoffrey, As Jorge mentioned, you can reference

Re: RecordPath...With INNER records

2021-03-25 Thread Mark Payne
Geoffrey, As Jorge mentioned, you can reference specific array indices in the RecordPath like /outer[0]/key, or you could reference all elements such as /outer[*]/key. But at this time, I don’t think LookupRecord will allow for multiple lookups per record. So the enrichment you’re looking at

Re: Writing custom logs

2021-03-25 Thread Etienne Jouvin
If you know all your tables, and there is too much you can duplicate this configuration for all messages. Le jeu. 25 mars 2021 à 13:03, Vibhath Ileperuma a écrit : > Hello Etienne, > > With the configuration you have mentioned, we can log messages related to > only one table using the

Re: Writing custom logs

2021-03-25 Thread Vibhath Ileperuma
Hello Etienne, With the configuration you have mentioned, we can log messages related to only one table using the LogMessage processor, can't we? But what I really want is to log custom messages related to all the tables using the LogMessage processor(s). But the messages should be written to

Re: [EXTERNAL] RE: ExtractText 1.13.1 only one regex evaluated on CSV file?

2021-03-25 Thread Chris Sampson
See the "Unsubscribe" links on the NiFi Mailing List page[1] [1] https://nifi.apache.org/mailing_lists.html --- *Chris Sampson* IT Consultant chris.samp...@naimuri.com On Thu, 25 Mar 2021 at 08:28, Aggarwal, Ashish wrote: > Team, > > > > What is the process to get

Re: Writing custom logs

2021-03-25 Thread Etienne Jouvin
Hello. What you can do it to prefix your log message, with something like [MY_TABLE] and then in the logback configuration use an EvaluatorFilter. Something like this : ${org.apache.nifi.bootstrap.config.log.dir}/loreal-sync-dctm-process.log

Writing custom logs

2021-03-25 Thread Vibhath Ileperuma
Hi All, I'm developing a NIFI flow which fetches data from a set of database tables. To log custom messages (no. of rows fetched, query execution time etc.), I'm using a 'LogMessage' processor. By changing the logback.xml file, I was able to write my custom messages in a separate log file. But

Re: [EXTERNAL] RE: ExtractText 1.13.1 only one regex evaluated on CSV file?

2021-03-25 Thread Aggarwal, Ashish
Team, What is the process to get removed from this distribution list. Thanks Ashish Aggarwal Data Engineer | DACI-Data Engineering | Lowe’s Companies, Inc. From: Hendrik Ruijter Date: Thursday, 25 March 2021 at 13:54 To: users@nifi.apache.org Subject: [EXTERNAL] RE: ExtractText 1.13.1 only

RE: ExtractText 1.13.1 only one regex evaluated on CSV file?

2021-03-25 Thread Hendrik Ruijter
Sorry! Test OK in 1.13.2. From: Joe Witt Sent: den 25 mars 2021 08:38 To: users@nifi.apache.org Subject: Re: ExtractText 1.13.1 only one regex evaluated on CSV file? Please avoid 1.13.1. Grab 1.13.2 On Wed, Mar 24, 2021 at 11:33 PM Hendrik Ruijter mailto:hendrik.ruij...@verisure.com>> wrote:

Re: ExtractText 1.13.1 only one regex evaluated on CSV file?

2021-03-25 Thread Joe Witt
Please avoid 1.13.1. Grab 1.13.2 On Wed, Mar 24, 2021 at 11:33 PM Hendrik Ruijter < hendrik.ruij...@verisure.com> wrote: > [Sending again since I cannot see this mail in the archive] > > Hello, ExtractText 1.12.1 evaluates the regex for each flowfile and adds > attributes to each flowfile.

Re: RecordPath...With INNER records

2021-03-25 Thread Jorge Machado
Hey Greene, The LookupRecord as a RecordPath as input. Check out this docs: https://nifi.apache.org/docs/nifi-docs/html/record-path-guide.html#structure or https://www.nifi.rocks/record-path-cheat-sheet/

RE: ExtractText 1.13.1 only one regex evaluated on CSV file?

2021-03-25 Thread Hendrik Ruijter
[Sending again since I cannot see this mail in the archive] Hello, ExtractText 1.12.1 evaluates the regex for each flowfile and adds attributes to each flowfile. However, ExtractText 1.13.1 appears to evaluate the regex for the first flowfile only adding the same attributes to all flowfiles.