Hi Stéphane, NiFi is intentionally designed to allow you to make changes on a very tight cycle while interacting with live data. This separates it from a number of tools that require lengthy deployment processes to push them to sandbox/production environments and test with real data. As one of the engineers says frequently, “NiFi is like digging irrigation ditches as the water flows, rather than building out a sprinkler system in advance."
Each processor component and queue will show statistics about the data they are processing and you can see further information and history by going into the Stats dialog of the component (available through the right-click context menu on each element). To monitor actual data (such as the response of an HTTP request or the result of a JSON split), I’d recommend using the LogAttribute processor [1]. You can use this processor to print the value of specific attributes, all attributes, flowfile payload, expression language queries, etc. to a log file and monitor the content in real-time. I’m not sure if we have a specific tutorial for this process, but many of the tutorials and videos [2] for other flows include this functionality to demonstrate exactly what you are looking for. If you decide that the flow needs modification, you can also replay flowfiles through the new flow very easily from the provenance view [3]. One other way to explore the processors, is of course to write a unit test and evaluate the result compared to your expected values. This is more advanced and requires downloading the source code and does not use the UI, but for some more complicated usages or for people familiar with the development process, NiFi provides extensive flow mocking capabilities to allow developers to do this. I hope this helps. If we can make anything clearer or you have more questions, please don’t hesitate to ask. [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.LogAttribute/index.html <https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.LogAttribute/index.html> [2] https://nifi.apache.org/videos.html <https://nifi.apache.org/videos.html> [3] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#replaying-a-flowfile <https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#replaying-a-flowfile> Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Apr 28, 2016, at 4:06 PM, Stéphane Maarek <stephane.maa...@gmail.com> > wrote: > > Hi, > > I'm very new to nifi and love the concept. As part of the process, I'm > learning. My biggest frustration is that I can't see the data flowing through > the system as I do development. > > Maybe I missed an article or a link, but is it possible to view the data > while in the flow? I.e. Say I create a get http, I'd like it to fire once, > get some data so I can see what it looks like. Then if I do a split json, I'd > like to see if my output of it is what I expected or if I somehow messed up, > etc etc > > I hope my question is clear > > Thanks in advance, > Stéphane
signature.asc
Description: Message signed with OpenPGP using GPGMail
