This looks promising, Lehel. Visiting my OpenSearch home page in the AWS dashboard, I do see that I have an IAM role associated with it. That role is AWSServiceRoleFoirAmazonOpenSearchService.
I select that role, but don't see that I have an ACCESS_KEY or SECRET_KEY associated with it. Its Type is AWS Managed. Looking at the top of the dashboard for this role, it appears that the role has a REGION of Global. My OpenSearch Service has a REGION of US East (N. Virginia). When I created my OpenSearch service, I did so only with a master user name and master user password. Should I instead explicitly create access and secret keys for my Amazon OpenSearch Service? Can you say a few words regarding how I get to these keys? On Sat, Oct 21, 2023 at 9:51 PM Lehel Boér <lehe...@hotmail.com> wrote: > Hi James, > > I'm not sure if the username/password authentication is enough in this > case. The AWS CLI automatically handles the authentication and > authorization for you, using the credentials you have configured for your > CLI. This looks like an authorization issue between curl and the AWS > service. > Curl supports *--aws-sigv-4* requests which you can use with the access > key of the IAM role set in OpenSearch. I managed to get it working for GET > requests. > https://how.wtf/aws-sigv4-requests-with-curl.html > > -XGET;https://$DOMAIN/$PATH;-H;'Content-Type: > application/json';--user;$ACCES_KEY:$SECRET_KEY;--aws-sigv4;aws:amz:$REGION:es > > Kind Regards, > Lehel > > ------------------------------ > *From:* James McMahon <jsmcmah...@gmail.com> > *Sent:* Saturday, October 21, 2023 20:25 > *To:* users <users@nifi.apache.org> > *Subject:* curl from ExecuteStreamCommand > > I have tested this curl from my ec2 command line: > curl -XPUT -u 'myusernm:myuserpw' ' > https://vpc-rampart-test-opensearch-nrqyb7jjpvmji6cp2qcvmyhcgq.us-east-1.es.amazonaws.com/movies/_doc/1' > -d '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, > "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], > "title": "Mars Attacks!"}' -H 'Content-Type: application/json' > > It successfully puts the json into my Amazon OpenSearch domain. > > That domain in the URL above is the Domain endpoint shown on the AWS > dashboard for its OpenSearch service. > > In NiFi the JSON is my flowfile content. I am trying to get my > ExecuteStreamCommand to run the curl, but it fails. NiFi indicates it gets > back this: > "message": "Request forbidden by administrative rules" > > This is how I have the processor configured. > Command Arguments - -XINPUT;-u;myusernm:myuserpw; > https://vpc-rampart-test-opensearch-nrqyb7jjpvmji6cp2qcvmyhcgq.us-east-1.es.amazonaws.com/movies/_doc/1;-H;'Content-Type: > application/json' > Command Path - /usr/bin/curl > Ignore STDIN - false > Argument Delimiter - ; > Max Attribute Length - 256 > > How can this be configured in the ExecuteStreamCommand processor to run > successfully? > > If the ExecuteStreamCommand executes the command just as if we were at the > command line, what is getting in the way here when I try to run this from > NiFi? >
