Thanks Andy. I think I should go for Jetty servlet as suggested in Koji's JIRA request " https://issues.apache.org/jira/browse/NIFI-3469";
Regards On Tue, Feb 28, 2017 at 12:09 PM, Andy LoPresto <alopre...@apache.org> wrote: > Mohammed, > > NiFi Input Ports are definitely not the answer you are looking for here. > As stated, they are specifically for NiFi to NiFi communication using Site > to Site protocol, either over raw sockets or HTTPS. They do not accept > arbitrary HTTP requests from other entities. > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Feb 28, 2017, at 12:04 PM, mohammed shambakey <shambak...@gmail.com> > wrote: > > Thanks Koji > > On Tue, Feb 28, 2017 at 8:44 AM, Koji Kawamura <ijokaruma...@gmail.com> > wrote: > >> Mohammed, >> >> NiFi REST API is for managing NiFi itself and data flow running on it, >> as Andy described earlier, it's not for data transfer. It does have >> endpoints to receive POST requests but those are for adding Processors >> or Templates ... etc. Please refer NiFi REST API docs for details. >> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html >> >> Thanks, >> Koji >> >> On Tue, Feb 28, 2017 at 9:25 PM, mohammed shambakey >> <shambak...@gmail.com> wrote: >> > Thank you all for your help >> > >> > I think Koji's thread at JIRA will be the best solution when it is >> > implemented >> > >> > Just a quick question, Andy mentioned REST APIs for NIFI. I didn't find >> much >> > examples about using NIFI REST APIs. Is it possible to POST the WEB UI >> > (including all inputs from multipart form and uploaded files in a POST >> > request to a NIFI input port of a remote NIFI instance? because, I >> think, >> > this will avoid using "handlehttprequest" processor and allow users to >> > communicate with the remote NIFI instance both from WEB UI and/or NIFI >> > instance (e.g., site-to-site). >> > >> > Regards >> > >> > On Mon, Feb 27, 2017 at 6:54 PM, Koji Kawamura <ijokaruma...@gmail.com> >> > wrote: >> >> >> >> If the multipart contains only text data, it might be possible to use >> >> those processors Andy suggested. However, if we support multipart >> >> natively, it can be done much easier and safer. I created a JIRA for >> >> this improvement before, but haven't been able to find a time to >> >> implement it myself. >> >> If anyone interested in tackling this enhancement, it'd be greatly >> >> appreciated. >> >> https://issues.apache.org/jira/browse/NIFI-3469 >> >> >> >> Thanks, >> >> Koji >> >> >> >> On Tue, Feb 28, 2017 at 4:59 AM, Andy LoPresto <alopre...@apache.org> >> >> wrote: >> >> > Absolutely. Use some combination of SplitText, SplitContent, >> SplitJSON, >> >> > and >> >> > ExtractText to split the content into multiple flowfiles on regular >> >> > expressions, new lines, and/or byte sequences and then >> RouteOnContent to >> >> > send the flowfiles which reference a file to a processor to >> >> > retrieve/parse >> >> > the file contents. >> >> > >> >> > Andy LoPresto >> >> > alopre...@apache.org >> >> > alopresto.apa...@gmail.com >> >> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> > >> >> > On Feb 27, 2017, at 2:08 PM, mohammed shambakey < >> shambak...@gmail.com> >> >> > wrote: >> >> > >> >> > This is an example of the output file from "handlehttprequest": >> >> > >> >> > " >> >> > -----------------------------831805444208556434014280960^M >> >> > Content-Disposition: form-data; name="userid"^M >> >> > ^M >> >> > jpl_user^M >> >> > -----------------------------831805444208556434014280960^M >> >> > Content-Disposition: form-data; name="scripts"; filename="a.py"^M >> >> > Content-Type: text/x-python^M >> >> > ^M >> >> > ^M >> >> > -----------------------------831805444208556434014280960^M >> >> > Content-Disposition: form-data; name="scripts"; filename="b.py"^M >> >> > Content-Type: text/x-python^M >> >> > ^M >> >> > ^M >> >> > -----------------------------831805444208556434014280960^M >> >> > Content-Disposition: form-data; name="scripts"; >> filename="conf.json"^M >> >> > Content-Type: application/json^M >> >> > ^M >> >> > { >> >> > "userid":"jpl_user_1", >> >> > "scripts":["a.py","b.py"] >> >> > } >> >> > ^M >> >> > -----------------------------831805444208556434014280960--^M >> >> > " >> >> > >> >> > Can I extract different components of this file without script at >> server >> >> > side? >> >> > >> >> > Regards >> >> > >> >> > On Mon, Feb 27, 2017 at 10:52 AM, Andy LoPresto < >> alopre...@apache.org> >> >> > wrote: >> >> >> >> >> >> Mohammed, >> >> >> >> >> >> So from what I understand you have a separate web application which >> >> >> accepts a variety of data (a combination of form fields and file >> >> >> uploads) >> >> >> and submits that (I’m assuming via POST) to NiFi. I’m not sure what >> >> >> problem >> >> >> you had extracting these “components” from the output flowfile from >> the >> >> >> HandleHTTPRequest processor. Can you change your web interface to >> send >> >> >> separate POST requests to NiFi and logically separate these >> components >> >> >> from >> >> >> the beginning? Can you modify the format of the request (i.e. make >> it >> >> >> JSON, >> >> >> etc.) so it is properly structured for NiFi to parse and operate on >> >> >> this >> >> >> data? >> >> >> >> >> >> NiFi treats the content of the flowfile the same once it is in the >> >> >> system >> >> >> regardless of the source processor, so you should be able to split, >> >> >> extract, >> >> >> or manipulate the contents of a single flowfile containing multiple >> >> >> form >> >> >> elements and file bodies. >> >> >> >> >> >> If you are referencing NiFi’s Input Port components as a destination >> >> >> for >> >> >> an arbitrary HTTP request, this will not work. The input ports are >> >> >> designed >> >> >> to work with NiFi’s Site to Site protocol (either over raw sockets >> or >> >> >> HTTP(S)). >> >> >> >> >> >> >> >> >> Andy LoPresto >> >> >> alopre...@apache.org >> >> >> alopresto.apa...@gmail.com >> >> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> >> >> >> >> On Feb 27, 2017, at 11:09 AM, mohammed shambakey < >> shambak...@gmail.com> >> >> >> wrote: >> >> >> >> >> >> Thanks Andy >> >> >> >> >> >> I have a user web interface that send some form information (e.g., >> user >> >> >> id), as well as upload multiple files, to a remote NIFI instance. >> When >> >> >> using >> >> >> "handlhttprequest" processor, it results in a single file that >> contains >> >> >> "form inputs" and bodies of uploaded files. >> >> >> >> >> >> I want to extract individual components of this file (i.e., download >> >> >> all >> >> >> uploaded files and extract other form information (e.g., userid) in >> a >> >> >> JSON >> >> >> or other text file. I had a similar problem before and I learnt that >> >> >> "httphandlerequest" has a problem to extract individual compoents >> from, >> >> >> so I >> >> >> have to write my own scripts to extract these components (form >> inputs >> >> >> and >> >> >> uploaded files). >> >> >> >> >> >> As NIFI input ports does not face this problem of >> "handlehttprequest", >> >> >> I >> >> >> though if I can send form inputs and upload files directly to "NIFI >> >> >> input >> >> >> port" at remote site, then I don't have to write my own scripts to >> >> >> extract >> >> >> individual components. >> >> >> >> >> >> Regards >> >> >> >> >> >> On Mon, Feb 27, 2017 at 10:02 AM, Andy LoPresto < >> alopre...@apache.org> >> >> >> wrote: >> >> >>> >> >> >>> Mohammed, >> >> >>> >> >> >>> I’m not sure I understand your question. NiFi has a REST API which >> >> >>> supports the UI and allows scripts/other applications to make >> requests >> >> >>> of >> >> >>> the platform. This runs on the input port specified in >> >> >>> nifi.properties. >> >> >>> >> >> >>> If you want to submit form data and uploaded files as data to be >> >> >>> ingested >> >> >>> by NiFi, HandleHTTPRequest or (less likely) ListenHTTP are the >> >> >>> processors >> >> >>> you would want to use. >> >> >>> >> >> >>> If you can clarify your use case, we should be able to offer more >> >> >>> assistance. >> >> >>> >> >> >>> Andy LoPresto >> >> >>> alopre...@apache.org >> >> >>> alopresto.apa...@gmail.com >> >> >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> >>> >> >> >>> On Feb 27, 2017, at 9:15 AM, mohammed shambakey < >> shambak...@gmail.com> >> >> >>> wrote: >> >> >>> >> >> >>> Hi >> >> >>> >> >> >>> Is it possible to send http request (including form data and >> uploaded >> >> >>> files) to NIFI input port instead of "HandleHTTPRequest" processor? >> >> >>> >> >> >>> Regards >> >> >>> >> >> >>> -- >> >> >>> Mohammed >> >> >>> >> >> >>> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Mohammed >> >> >> >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Mohammed >> >> > >> >> > >> > >> > >> > >> > >> > -- >> > Mohammed >> > > > > -- > Mohammed > > > -- Mohammed
