good to know, thanks Bryan. My use case is similar to Shawn's when external consumers can request data from engines like Hive or Impala. I will play with it a bit more. I guess another problem would be a pagination of the response payload.
On Fri, May 4, 2018 at 9:18 AM, Bryan Bende <bbe...@gmail.com> wrote: > I think the point of the original thread was that building a REST > service in NiFi wasn't necessarily meant to be used as the back-end of > a full-blown web application that is used by hundreds/thousands of > users. > > However there are still plenty of situations where it is certainly > fine, especially for creating a mechanism for system-to-system > communication where there was no previous way to access the data > easily, and now it can be wrapped in a REST API for some other system > to call. > > On Thu, May 3, 2018 at 9:28 PM, Boris Tyukin <bo...@boristyukin.com> > wrote: > > Hi Joe, > > > > Here is that thread. I just read it again and I probably asked a pretty > > broad question though but I also wanted to create a rest endpoint through > > nifi to pull data from persistent storage > > > > http://apache-nifi.1125220.n5.nabble.com/NiFi-as-a-Web- > Services-server-td21938.html > > > > On Thu, May 3, 2018, 16:18 Joe Witt <joe.w...@gmail.com> wrote: > >> > >> Boris, > >> > >> can you share more about the context in which you were told it would > >> be a bad idea? There might be an important clarifying point there. > >> The pattern is certainly useful so we want to get the right messaging > >> around it. > >> > >> Thanks > >> > >> On Thu, May 3, 2018 at 4:17 PM, Shawn Weeks <swe...@weeksconsulting.us> > >> wrote: > >> > The rest api I'm building is primarily for generating data extracts > >> > using > >> > curl or wget and doesn't have the performance requirements that an > >> > Angular > >> > Rest UI would require. I haven't had any trouble syncing requests and > >> > responses as the Handle HTTP Request and Response Processors manage > that > >> > with an HTTP Context Map. You need to make sure you don't lose the > >> > attribute > >> > it's using and you can't return multiple responses to the same request > >> > as > >> > the first response closes the request. > >> > > >> > > >> > Thanks > >> > > >> > Shawn > >> > > >> > ________________________________ > >> > From: Boris Tyukin <bo...@boristyukin.com> > >> > Sent: Thursday, May 3, 2018 2:57:37 PM > >> > To: users@nifi.apache.org > >> > Subject: Re: Fetch Contents of HDFS Directory as a Part of a Larger > Flow > >> > > >> > Shawn, I am not answering your question but I am curious how you would > >> > see > >> > logistics of HTTP Request and Response Processors. Is your intent to > >> > build > >> > REST api entirely in NiFi? I asked a question here before as I had a > >> > similar > >> > idea but was told it was not a good strategy as it would be difficult > to > >> > sync request/response. > >> > > >> > On Thu, May 3, 2018 at 11:35 AM, Shawn Weeks < > swe...@weeksconsulting.us> > >> > wrote: > >> > > >> > I'm building a rest service with the HTTP Request and Response > >> > Processors to > >> > support data extracts from Hive. Since some of the extracts can be > quiet > >> > large using the SelectHiveQL Processor isn't a performant option and > >> > instead > >> > I'm trying to use on demand Hive Temporary Tables to do the heavy > >> > lifting > >> > via CTAS(Create Table as Select). Since GetHDFS doesn't support an > >> > incoming > >> > connection I'm trying to figure out another way to fetch the files > Hive > >> > creates and return them as a download in the web service. Has anyone > >> > else > >> > worked out a good solution for fetching the contents of a directory > from > >> > HDFS as a part of larger flow? > >> > > >> > > >> > Thanks > >> > > >> > Shawn > >> > > >> > >
