Re: NiFi 2.0.0-M2

[Russell Bateman]

Don't know if this is relevant let along helpful, but we were on CentOS 
for many years and, when it died, we moved to Alma.


Best of luck to you,

Russ

On 3/5/24 07:56, Marton Szasz wrote:
CentOS 7 is a very old distribution, so you may run into issues, but 
in theory, if you can install Java 21 on it, and start NiFi using 
that, then it should work.


I'd be surprised if Java 21 was included in the CentOS 7 package 
repositories. You'll most likely need to install it manually, and set 
JAVA_HOME appropriately in the environment you're using to start NiFi.


Marton

On 3/5/24 14:10, Pierre Villard wrote:

Hi,

NiFi is OS-agnostic for linux based distributions. You can download the
tar.gz from the download page of the website and follow the 
instructions to

get started on CentOS 7.

HTH,
Pierre

Le mar. 5 mars 2024 à 14:07, Gleb Efimov  a 
écrit :



Good afternoon I'm trying to deploy NiFi 2.0.0-M2 on Centos 7, but,
unfortunately, I can't find a tar archive with the distribution I need.
Could you tell me where I can get it?
And is it even possible to deploy NiFi 2.0.0-M2 on Centos 7?
Thank you very much.

Sincerely, Efimov Gleb.

Re: QueryRecord processor fails with use of CONTAINS_SUBSTR

[Pierre Villard]

Hi Dan,

As you can see on the doc there is a 'b' next to CONTAINS_SUBSTR and the
documentation explains its meaning:

The ‘C’ (compatibility) column contains value:
> ‘*’ for all libraries,
> ‘b’ for Google BigQuery (‘fun=bigquery’ in the connect string),
> ‘c’ for Apache Calcite (‘fun=calcite’ in the connect string),
> ‘h’ for Apache Hive (‘fun=hive’ in the connect string),
> ‘m’ for MySQL (‘fun=mysql’ in the connect string),
> ‘q’ for Microsoft SQL Server (‘fun=mssql’ in the connect string),
> ‘o’ for Oracle (‘fun=oracle’ in the connect string),
> ‘p’ for PostgreSQL (‘fun=postgresql’ in the connect string),
> ’s’ for Apache Spark (‘fun=spark’ in the connect string).


I believe we only support the ones for Apache Calcite. This is probably
something we could/should improve.

HTH,
Pierre


Le mer. 6 mars 2024 à 18:14, Dan S  a écrit :

> I am trying to use the QueryRecord processor with a SQL statement similar
> to:
>
> SELECT mi FROM FLOWFILE WHERE CONTAIN_SUBSTR(somedetails, 'Fred')
>
> which fails with the following error message in the logs:
> Caused by org.apache.calcite.runtime.CalciteContextException: From column
> 31 to line 1, column 66: No match found for function signature
> CONTAINS_SUBSTR(,  )
>
> In the Apache Calcite documentation
> I see CONTAINS_SUBSTR
> defined. Why cannot I use it in NIFI? Are there limitations of what
> functions that can be used from Apache Calcite?
>

QueryRecord processor fails with use of CONTAINS_SUBSTR

[Dan S]

I am trying to use the QueryRecord processor with a SQL statement similar
to:

SELECT mi FROM FLOWFILE WHERE CONTAIN_SUBSTR(somedetails, 'Fred')

which fails with the following error message in the logs:
Caused by org.apache.calcite.runtime.CalciteContextException: From column
31 to line 1, column 66: No match found for function signature
CONTAINS_SUBSTR(,  )

In the Apache Calcite documentation
I see CONTAINS_SUBSTR
defined. Why cannot I use it in NIFI? Are there limitations of what
functions that can be used from Apache Calcite?

Re: [DISCUSS] New schema repository idea (with proof of concept)

[David Handermann]

Mike,

Thanks for raising this question, and providing the example repository.

Although it sounds like a POJO-based repository could be useful in
some cases, it does not seem like something that should be included
for community support.

Part of the value of a Schema Registry is a shared location for data
description. Although supporting property or file-based Schema
Registries is useful in NiFi itself, the general pattern is
externalized storage and maintenance of schema definitions.

>From another angle, this could be similar to code-first versus
contract-first API development. Each approach has its positives and
negatives. When it comes to a Schema Registry, however, it seems like
the contract needs to be defined outside of code.

Introspecting JAR files also raises questions about what to include or
exclude, and how to handle edge cases for certain class definitions.
This seems like the more significant problem. For this reason, it
seems better to rely on external operations to produce Avro schema
definitions, rather than supporting that in NiFi itself.

Those are my initial thoughts, perhaps others can provide additional
perspective.

Regards,
David Handermann

On Sat, Mar 2, 2024 at 9:18 AM Mike Thomsen  wrote:
>
> I've had this project on the back burner for a while and wanted to share it
> with the team. It's a schema repository implementation that is designed to
> take a JAR file with POJOs and use Jackson's schema generation API to
> generate Avro schemas from those on startup. It also uses (via Jackson)
> Avro annotations to help specify particular implementation details where
> necessary. The code can be found here. Haven't worked on it lately, but it
> should easily run on 1.25:
>
> https://github.com/MikeThomsen/nifi-pojo-schema-repository-bundle
>
> I am planning to get the repo ready for a PR unless someone raises reasons
> why including it might be a poor fit. I think for a lot of teams this might
> be a killer feature because it would allow them to use Avro with existing
> enterprise POJOs and stuff like that without having to write them by hand.
>
> Thoughts?

RE: Nifi flow json to MiNifi flow json 2.0.0.M2 NIFI-12644

[Premarajan, Ajay]

Hi Team,

The nifi flow files when downloaded does not contain sensitive property 
information.
https://issues.apache.org/jira/browse/NIFI-12644 was addressed to convert nifi 
flow json to Minifi flow json. Since the sensitive properties are not present 
in the nifi flow files, we have to add it manually before starting MiNifi.

What is the way to handle this scenario, will senstivie properties be added to 
the Nifi flow files when downloaded or is there any approach for this?

I am enquiring this because we are doing an integration of MiNifi for internal 
project. I would be happy to contribute for the solution.

Kind Regard,
Ajay Premarajan


From: Premarajan, Ajay
Sent: Friday, March 1, 2024 7:31 AM
To: dev@nifi.apache.org
Subject: Nifi flow json to MiNifi flow json 2.0.0.M2

Hi Team,

I have a doubt regarding importing Nifi json flow to MiNifi json flow.
I am using code from https://github.com/apache/nifi/pull/8393 which was used 
resolve https://issues.apache.org/jira/browse/NIFI-12644 issue.

I have noticed that sensitive properties are not present when nifi flow json is 
downloaded. How are these values passed to MiNifi for operation.

For  ex:
I have an InvokeHttp processor which uses SSL controller and has the attribute 
“Request Username” and “Request Password” configured. When I download the Nifi 
Flow json, the “Request Password” field is missing. Plus the SSL controller 
donot have the password information in them.

this is causing problem when I try to convert the Nifi flow json into MiNifi 
flow json using the pull request code.

Please suggest an way forward.


Regards,
Ajay Premarajan