Wyllys, I am sorry but I am having a different experience when I try to reproduce the scenario you describe.
I set up a sample flow which uses an InvokeHTTP processor to send a flowfile to a remote HTTP endpoint. That endpoint happens to be a disparate flow in NiFi because it was easy to configure on the fly, but it could be any remote web server. I can verify that when I send flowfile content, it is not transmitting the flowfile attributes unless I explicitly configure it to. Please see below an annotated excerpt from the nifi-app.log file and a link to the template I exported with this flow. I think the issue may be one of terminology? Attributes and content are separate in NiFi and flowfiles distinguish very cleanly between the two. Are you referring to elements of the JSON body as “attributes”? In that case, you would need to use a ReplaceText, EvaluateJsonPath, JoltTransformJson, or other processor as Jerry suggested to extract/manipulate the JSON in the flowfile content to the exact data you wish to send over HTTP before using InvokeHTTP. If you want to do this but not lose the other content, you can split the flow by dragging an additional “success” relationship from the preceding processor to handle the two different flow behaviors. Template for flow (tested on NiFi 1.9.2): https://gist.github.com/alopresto/0be1cf65ba70d5d36aeae1e6f2f60702 <https://gist.github.com/alopresto/0be1cf65ba70d5d36aeae1e6f2f60702> Log output from flow: https://gist.github.com/alopresto/b596372005032b9c80063a0652a955ea Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jun 18, 2019, at 7:47 PM, Jerry Vinokurov <grapesmo...@gmail.com> wrote: > > Hi Wyllys, > > One way that I solve this problem in my work is to use the AttributesToJSON > processor to form the body of the POST before sending it. Granted, that does > override the body of the flowfile, so I'm not sure if that works for your > specific case, but it does allow you to select which attributes you want to > turn into JSON key/value pairs. For more complex formations I would suggest > checking out the JOLT Transform processor, which can be quite powerful, if > somewhat painful to work with. > > Jerry > > On Tue, Jun 18, 2019 at 9:49 PM Wyllys Ingersoll > <wyllys.ingers...@keepertech.com <mailto:wyllys.ingers...@keepertech.com>> > wrote: > Andy - > Yes, Im referring to the PUT and POST methods, in which case the processor > just sends the entire flowfile as a JSON object in the message body. I'd > prefer to either have the option to exclude some of the flow attributes or > (even better) have the ability to craft my own message body. There are lots > of instances where the receiver of the PUT or POST expects a particular > structure that doesn't easily work with just a flat JSON-ified set of flow > attributes. > > One example: > We have a flow that has an authentication token as one of the attributes > and a bunch of other key/value pairs used for other purposes. In the > InvokeHTTP processor, I use the auth token attribute in an Authorization > header by creating a dynamic attribute with the correct format > ("Authorization: Bearer ${token}"). However, the recipient of the PUT/POST > also expects the body of the request to be formatted a specific way and does > not expect or want to see the auth token or some of the other unrelated > information that ends up getting transmitted as part of the message body > simply because they are flow attributes. So, if InvokeHTTP were able to > exclude certain fields from the message body AND also allow the body of the > message to be configurable into a structure other than just a flat dictionary > of flow attributes, it would be much more powerful and useful. As it stands, > I'm thinking I may have to develop a custom processor to get past this issue, > which is not ideal at all. > > Thanks! > Wyllys Ingersoll > > > > > On Tue, Jun 18, 2019 at 8:34 PM Andy LoPresto <alopre...@apache.org > <mailto:alopre...@apache.org>> wrote: > Hi Wyllys, > > Sorry to hear you are having trouble with this processor. Can you please > provide a more detailed example of an incoming flowfile and what your > expected output is compared to what is currently happening? Based on my > understanding, the flowfile attributes are only sent as request headers, and > that can be controlled using the regular expression value of “Attributes to > Send”. I believe only the flowfile content is sent as the request body when > using PUT or POST. Thanks. > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jun 18, 2019, at 3:01 PM, Wyllys Ingersoll >> <wyllys.ingers...@keepertech.com <mailto:wyllys.ingers...@keepertech.com>> >> wrote: >> >> >> it would be nice to be able to exclude attributes from the message body when >> doing a PUT or POST in the invokeHTTP processor. Currently, there's no way >> to selectively choose which attributes go into the message body without >> using a replaceText processor in front of it, but that completely removes >> those attributes from being used later which is not always the desirable. >> >> There are lots of situations where it would be nice to be able to use some >> flow attributes just in the request header or for other parts of the >> processor configuration without including them in the message body itself. >> For example, taking an authentication token that is carried along as a flow >> attribute so it can be used in an authorization header and NOT included in >> the body of the request. >> >> In general, invokeHTTP needs to allow for more control over the format and >> content of the message body being sent. >> >> -Wyllys Ingersoll >> > > > > -- > http://www.google.com/profiles/grapesmoker > <http://www.google.com/profiles/grapesmoker>
