Re: How to access to an Amazon API?

[Otto Fowler]

That error means that we passed info into AWS api ( asked the region to find 
the resource ) and it said nope, and tried to guess.
I would have to try to setup the processor with your configuration ( enough to 
try to parse, ok if it fails later because of cred or whatever ).


> On Jun 15, 2021, at 06:26, Luca Giovannini  
> wrote:
> 
>  
> Thank you Otto,
>  
> I can confirm that the resource API is correct: I tested it successfully with 
> Postman by doing a GET on the request URL 
> (https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/prod/test 
> ) and 
> providing AccessKey, SecretKey and AWS Region in the “Authorization” tab of 
> the Postman client.
>  
> I also checked log/nifi-app.log and the only informative line that I found is:
> 2021-06-15 11:59:24,046 INFO [Timer-Driven Process Thread-8] 
> c.a.i.DefaultServiceEndpointBuilder {execute-api, eu-west-1} was not found in 
> region metadata, trying to construct an endpoint using the standard pattern 
> for this region: 'execute-api.eu-west-1.amazonaws.com 
> '.
> Which seems to indicate that the processor is not using the endpoint I 
> provided as parameter 
> (https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/) 
> ...
>  
> I also checked the provenance data (both from the “Provenance” button of the 
> output FlowFile listing and from the “Data Provenance” element of the main 
> menu) for records of the call, but I could not find any provenance data at 
> all for the processor InvokeAWSGatewayApi under scrutiny (my flow right now 
> is made up just by that processor alone and a LogMessage one).
>  
> Is this new information shedding any more light on the problem for you?
>  
> Thank you very much,
> Luca
>  
>  
> Luca Giovannini
> Information Systems Analyst
> Dedagroup Public Services
> www.linkedin.com/in/lucagio/ 
> T +39.051.278.928 | M +39.347.799.3183 | VoIP 951.128
> Dedagroup Public Services Srl – Sede di Casalecchio di Reno, Via del Lavoro 67
> www.dedagroup.it/public-services 
>  
>  
>  
> Da: Otto Fowler mailto:ottobackwa...@gmail.com>> 
> Inviato: lunedì 14 giugno 2021 19:48
> A: users@nifi.apache.org 
> Oggetto: Re: How to access to an Amazon API?
>  
> **ATTENZIONE** Questo messaggio proviene da un ACCOUNT ESTERNO, presta 
> attenzione ad eventuali link o allegati al suo interno.
>  
> You should also confirm the resource api you are trying to call is correct.
>  
> On Jun 14, 2021, at 12:04, Luca Giovannini  > wrote:
>  
>  
> Hello everybody!
>  
> I have no experience with Amazon services but I now have to access to an API 
> hosted on AWS.
> I have been given the following parameters:
> HTTP Method  GET
> Amazon Region   eu-west-1
> Access Key ID    (hidden 
> but known)
> Secret Access Key    (hidden but 
> known)
> Amazon Gateway Api ResourceName/test
> Amazon Gateway Api Endpoint 
> https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/ 
> 
>  (anonymized)
>  
> I have used the “InvokeAWSGatewayApi” and set the previous 6 parameters (and 
> only those), but I got {"message": "Internal server error"} as an answer.
> Maybe I did not format the ResourceName correctly (I see here [*] that the 
> ResourceName is usually much longer)?
> I am using NiFi 1.12.1.
>  
> Thank you very much for your help!
>  
> Luca Giovannini
>  
> [*] 
> https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html 
> 
>  
>  
> Luca Giovannini
> Information Systems Analyst
> Dedagroup Public Services
> www.linkedin.com/in/lucagio/ 
> 

Regarding jira issue: NIFI-7856

[sanjeet rath]

Hi,

The symptoms mentioned in the jira issue(
*https://issues.apache.org/jira/browse/NIFI-7856
 *), i am
observing this in one of our PROD clusters.

ERROR [Compress Provenance Logs-1-thread-2]
o.a.n.p.s.EventFileCompressor Failed to compress
./provenance_repository/1693519.prov on rollover
java.io.FileNotFoundException: ./provenance_repository/1693519.prov
(No such file or directory)


i saw the code is fixed in 1.13 version with bellow file changes in
nifi-provenance-repository-bundle .

nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/store/RecordWriterLease.java

diff

 | blob

 | history

nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/store/WriteAheadStorePartition.java

So i have modified above 2 file changes on top of my 1.12.1 version of my
nifi-provenance-repository-bundle and build the nifi-provenance-repository.
nar file. then i will deploy this new Nar file to to /Lib folder of 1.12.1
version of niif

Is my above approach is correct ?

Second thing is, I am facing one issue in 1.12 version, unable to replicate
the provenance error in the lower environment (Tried with the 7856.xml
template attached in the jira by Mark)
So not able to understand whether the above change i made is worked or not.
As i can not directly deploy the Nar to prod env where the error is
constantly coming in every hour.

Could you please help me to replicate this issue in 1.12.1 version
Along with the template any other config changes i need to do replicate the
issue.


Regards,
-- 
Sanjeet Kumar Rath,

R: How to access to an Amazon API?

[Luca Giovannini]

Thank you Otto,

I can confirm that the resource API is correct: I tested it successfully with 
Postman by doing a GET on the request URL 
(https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/prod/test) and 
providing AccessKey, SecretKey and AWS Region in the "Authorization" tab of the 
Postman client.

I also checked log/nifi-app.log and the only informative line that I found is:
2021-06-15 11:59:24,046 INFO [Timer-Driven Process Thread-8] 
c.a.i.DefaultServiceEndpointBuilder {execute-api, eu-west-1} was not found in 
region metadata, trying to construct an endpoint using the standard pattern for 
this region: 'execute-api.eu-west-1.amazonaws.com'.
Which seems to indicate that the processor is not using the endpoint I provided 
as parameter (https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/)...

I also checked the provenance data (both from the "Provenance" button of the 
output FlowFile listing and from the "Data Provenance" element of the main 
menu) for records of the call, but I could not find any provenance data at all 
for the processor InvokeAWSGatewayApi under scrutiny (my flow right now is made 
up just by that processor alone and a LogMessage one).

Is this new information shedding any more light on the problem for you?

Thank you very much,
Luca


Luca Giovannini
Information Systems Analyst
Dedagroup Public Services
www.linkedin.com/in/lucagio/
T +39.051.278.928 | M +39.347.799.3183 | VoIP 951.128
Dedagroup Public Services Srl - Sede di Casalecchio di Reno, Via del Lavoro 67
www.dedagroup.it/public-services

[cid:image001.jpg@01D761E0.29244390]

Da: Otto Fowler 
Inviato: lunedì 14 giugno 2021 19:48
A: users@nifi.apache.org
Oggetto: Re: How to access to an Amazon API?

**ATTENZIONE** Questo messaggio proviene da un ACCOUNT ESTERNO, presta 
attenzione ad eventuali link o allegati al suo interno.

You should also confirm the resource api you are trying to call is correct.

On Jun 14, 2021, at 12:04, Luca Giovannini 
mailto:luca.giovann...@dedagroup.it>> wrote:


Hello everybody!

I have no experience with Amazon services but I now have to access to an API 
hosted on AWS.
I have been given the following parameters:

  *   HTTP Method  GET
  *   Amazon Region   eu-west-1
  *   Access Key ID    
(hidden but known)
  *   Secret Access Key    (hidden but 
known)
  *   Amazon Gateway Api ResourceName/test
  *   Amazon Gateway Api Endpoint 
https://zsslnu1awf.execute-api.eu-west-1.amazonaws.com/
 (anonymized)

I have used the "InvokeAWSGatewayApi" and set the previous 6 parameters (and 
only those), but I got {"message": "Internal server error"} as an answer.
Maybe I did not format the ResourceName correctly (I see here [*] that the 
ResourceName is usually much longer)?
I am using NiFi 1.12.1.

Thank you very much for your help!

Luca Giovannini

[*] 
https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html


Luca Giovannini
Information Systems Analyst
Dedagroup Public Services
www.linkedin.com/in/lucagio/
T +39.051.278.928 | M +39.347.799.3183 | VoIP 951.128
Dedagroup Public Services Srl - Sede di Casalecchio di Reno, Via del Lavoro 67

Re: Using a corporate SSL signed certificate

[Jens M. Kofoed]

Hi Emmanuel

I don't use the toolkit, I just do it manually.
I have found that a normal server certificate, generated i Microsoft
Windows is not working. The certificate for NiFi servers has to be both
serverAuth and clientAuth. So I have created a new certificate profil in
our PKI server for NiFi servers.
Next I create a server certificate for node1, with the following settings
Common name = node1.domain.net
alternative names:
dns = node1.domain.net
dns = clustername.domain.net

When I export the certificate (as a pfx file) I export it with the private
key and protect it with a password. I all so export my CA and ICA
certificates and copies all the the node1 server
To create the keystore file I use the following command:
keytool -importkeystore -destkeystore keystore.jks -srcstoretype PKCS12
-deststoretype jks -srckeystore node1.domain.net.pfx
Here you will have to provide the password for the certificate and set a
password for the keystore. I use the same password for both.

To create a truststore I use the folloing commands:
keytool -keystore truststore.jks -storetype jks -importcert
-trustcacerts -file CA.domain.net.cer -alias CA-DOMAIN
keytool -keystore truststore.jks -importcert -file ICA.domain.net.cer
-alias ICA-DOMAIN
You will have to provide a password for the truststore.

Now you will have to manually edit the nifi.properties file for the path to
the files and the passwords.

Just repeat the steps above for the other nodes. Keep in mind if you later
will use a StandardSSLContextService and use the keystore on each node, the
password for the certificate and keystores has to be the same.

For accessing my secure nifi cluster afterwards, I simple create a user
certificate to my self in windows. and configure the authorizers.xml with
the certificate name "CERTIFICATE"
Keep in mind that NiFi is case sentitive. Therefore I use identity mappings
in the nifi.properties file
nifi.security.identity.mapping.pattern.dn=^(.*)$
nifi.security.identity.mapping.value.dn=$1
nifi.security.identity.mapping.transform.dn=LOWER

This works fine for my.

Kind regards
Jens M. Kofoed


Den man. 14. jun. 2021 kl. 15.39 skrev QUEVILLON EMMANUEL - EXT-SAFRAN
ENGINEERING SERVICES (SAFRAN) :

> Hi list,
>
>
>
> We are trying to set a nifi secure installation using a SSL singed
> certificate by our corporate CA.
>
> This SSL certificate is signed for a domain name we’d like to use to
> access our nifi server(s).
>
> We’ve been unable to create a new certificate for our server using
> tls-toolkit for the main admin user identity to connect.
>
> 1)  Is it possible to use such SSL signed certificate to create a new
> one with tls-toolkit?
>
>
>
> We’ve followed this documentation
> https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_intermediate_ca
> and copied respective files and key to the right location and ran
> tls-toolkit command. However, tls-toolkit throw error complaining “The
> signing certificate was not signed by any known certificates
>
> ”
>
>
>
> We’ve also tried with the full chain certificate as an additional
> certificate file (option –additionalCACertificate), but it looks like
> tls-toolkit does not find all the certificate chain and stop at first level
> of the chain.
>
>
>
> Is anyone faced the same problematic?
>
> Any help or advice will be appreciated.
>
>
>
> Thanks, regards
>
>
>
> Emmanuel
>
> C2 - Restricted
>
>
> #
> " Ce courriel et les documents qui lui sont joints peuvent contenir des
> informations confidentielles, être soumis aux règlementations relatives au
> contrôle des exportations ou ayant un caractère privé. S'ils ne vous sont
> pas destinés, nous vous signalons qu'il est strictement interdit de les
> divulguer, de les reproduire ou d'en utiliser de quelque manière que ce
> soit le contenu. Toute exportation ou réexportation non autorisée est
> interdite Si ce message vous a été transmis par erreur, merci d'en informer
> l'expéditeur et de supprimer immédiatement de votre système informatique ce
> courriel ainsi que tous les documents qui y sont attachés."
> **
> " This e-mail and any attached documents may contain confidential or
> proprietary information and may be subject to export control laws and
> regulations. If you are not the intended recipient, you are notified that
> any dissemination, copying of this e-mail and any attachments thereto or
> use of their contents by any means whatsoever is strictly prohibited.
> Unauthorized export or re-export is prohibited. If you have received this
> e-mail in error, please advise the sender immediately and delete this
> e-mail and all attached documents from your computer system."
> #
>