OK, that helps. I put AttributesToJSON and a ReplaceText processor in front of my invokeHTTP and was able to get it to work as I wanted.
One of the confusing issues is that InvokeHTTP does not have any configuration that allows me to modify the flowfile content, which forces me to use those additional processors to make sure the content is correct. It might be a little more convenient if invokeHTTP had a couple of parameters to allow for setting the content format so that the extra processors are not necessary. On Tue, Jun 18, 2019 at 11:01 PM Andy LoPresto <alopre...@apache.org> wrote: > Wyllys, > > I am sorry but I am having a different experience when I try to reproduce > the scenario you describe. > > I set up a sample flow which uses an InvokeHTTP processor to send a > flowfile to a remote HTTP endpoint. That endpoint happens to be a disparate > flow in NiFi because it was easy to configure on the fly, but it could be > any remote web server. > > I can verify that when I send flowfile content, it is not transmitting the > flowfile attributes unless I explicitly configure it to. Please see below > an annotated excerpt from the nifi-app.log file and a link to the template > I exported with this flow. > > I think the issue may be one of terminology? Attributes and content are > separate in NiFi and flowfiles distinguish very cleanly between the two. > Are you referring to elements of the JSON body as “attributes”? In that > case, you would need to use a ReplaceText, EvaluateJsonPath, > JoltTransformJson, or other processor as Jerry suggested to > extract/manipulate the JSON in the flowfile content to the exact data you > wish to send over HTTP before using InvokeHTTP. If you want to do this but > not lose the other content, you can split the flow by dragging an > additional “success” relationship from the preceding processor to handle > the two different flow behaviors. > > Template for flow (tested on NiFi 1.9.2): > https://gist.github.com/alopresto/0be1cf65ba70d5d36aeae1e6f2f60702 > > Log output from flow: > https://gist.github.com/alopresto/b596372005032b9c80063a0652a955ea > > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Jun 18, 2019, at 7:47 PM, Jerry Vinokurov <grapesmo...@gmail.com> > wrote: > > Hi Wyllys, > > One way that I solve this problem in my work is to use the > AttributesToJSON processor to form the body of the POST before sending it. > Granted, that does override the body of the flowfile, so I'm not sure if > that works for your specific case, but it does allow you to select which > attributes you want to turn into JSON key/value pairs. For more complex > formations I would suggest checking out the JOLT Transform processor, which > can be quite powerful, if somewhat painful to work with. > > Jerry > > On Tue, Jun 18, 2019 at 9:49 PM Wyllys Ingersoll < > wyllys.ingers...@keepertech.com> wrote: > >> Andy - >> Yes, Im referring to the PUT and POST methods, in which case the >> processor just sends the entire flowfile as a JSON object in the message >> body. I'd prefer to either have the option to exclude some of the flow >> attributes or (even better) have the ability to craft my own message body. >> There are lots of instances where the receiver of the PUT or POST expects a >> particular structure that doesn't easily work with just a flat JSON-ified >> set of flow attributes. >> >> One example: >> We have a flow that has an authentication token as one of the >> attributes and a bunch of other key/value pairs used for other purposes. >> In the InvokeHTTP processor, I use the auth token attribute in an >> Authorization header by creating a dynamic attribute with the correct >> format ("Authorization: Bearer ${token}"). However, the recipient of the >> PUT/POST also expects the body of the request to be formatted a specific >> way and does not expect or want to see the auth token or some of the other >> unrelated information that ends up getting transmitted as part of the >> message body simply because they are flow attributes. So, if InvokeHTTP >> were able to exclude certain fields from the message body AND also allow >> the body of the message to be configurable into a structure other than just >> a flat dictionary of flow attributes, it would be much more powerful and >> useful. As it stands, I'm thinking I may have to develop a custom >> processor to get past this issue, which is not ideal at all. >> >> Thanks! >> Wyllys Ingersoll >> >> >> >> >> On Tue, Jun 18, 2019 at 8:34 PM Andy LoPresto <alopre...@apache.org> >> wrote: >> >>> Hi Wyllys, >>> >>> Sorry to hear you are having trouble with this processor. Can you please >>> provide a more detailed example of an incoming flowfile and what your >>> expected output is compared to what is currently happening? Based on my >>> understanding, the flowfile attributes are only sent as request headers, >>> and that can be controlled using the regular expression value of >>> “Attributes to Send”. I believe only the flowfile content is sent as the >>> request body when using PUT or POST. Thanks. >>> >>> >>> Andy LoPresto >>> alopre...@apache.org >>> *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>> On Jun 18, 2019, at 3:01 PM, Wyllys Ingersoll < >>> wyllys.ingers...@keepertech.com> wrote: >>> >>> >>> it would be nice to be able to exclude attributes from the message body >>> when doing a PUT or POST in the invokeHTTP processor. Currently, there's >>> no way to selectively choose which attributes go into the message body >>> without using a replaceText processor in front of it, but that completely >>> removes those attributes from being used later which is not always the >>> desirable. >>> >>> There are lots of situations where it would be nice to be able to use >>> some flow attributes just in the request header or for other parts of the >>> processor configuration without including them in the message body itself. >>> For example, taking an authentication token that is carried along as a flow >>> attribute so it can be used in an authorization header and NOT included in >>> the body of the request. >>> >>> In general, invokeHTTP needs to allow for more control over the format >>> and content of the message body being sent. >>> >>> -Wyllys Ingersoll >>> >>> >>> > > -- > http://www.google.com/profiles/grapesmoker > > >
