Re: How to use ListenHTTP processor?

2016-06-10 Thread Huagen peng
Thanks Mark. I am getting the results now. Huagen On Fri, Jun 10, 2016 at 4:40 PM, Mark Payne wrote: > Huagen, > > That property allows you to specify a Regular Expression that it will > match against all HTTP Header names. > If a header name matches the regex, an

Re: How to use ListenHTTP processor?

2016-06-10 Thread Mark Payne
Huagen, That property allows you to specify a Regular Expression that it will match against all HTTP Header names. If a header name matches the regex, an attribute will be added to the FlowFile with that name and value. For example, if your HTTP Headers look like: Content-Type:

How to use ListenHTTP processor?

2016-06-10 Thread Huagen peng
Hi, The ListenHTTP processor has a configuration “HTTP Headers to receive as Attributes (Regex)”. I tried many ways in vain to get some attributes in. Does anyone know how to get attributes directly in? I cannot find any example on it. I can post data in JSON format and use the

Re: How to effectively log the data flow in NiFi?

2016-06-10 Thread Huagen peng
Mark, your answer definitely helps. If I understand it correctly, you suggest that we change the default logging level, and the level of "org.apache.nifi.processors" to WARN, so that we will get a leaner log. On the data provenance, I really like what NiFi offers. The question is whether there

Re: Failure when running a workflow created from a template from another NiFi version.

2016-06-10 Thread James Wing
Thanks, Keith, the date is probably very helpful, I don't think we capture anything more specific in the build, like the last commit hash. It does continue the mystery as to the cause of the error you saw, and how exactly it might have been fixed. James On Fri, Jun 10, 2016 at 9:57 AM, Keith

Expression language toDate issue

2016-06-10 Thread Igor Kravzov
Hi, I have a date which comes in format like this one 2016-05-17T18:10:44-04:00. -04:00 is timezone. What format should I set in toDate method to properly convert to a date?

Re: How to effectively log the data flow in NiFi?

2016-06-10 Thread Mark Payne
Hi Huagen, This is typically the type of logging you will see in NiFi. Each processor will generally log at an INFO level what it is doing for each FlowFile. Unfortunately, though, this can become extremely verbose, and many people want that logging toned down, so in the master branch of NiFi,

How to effectively log the data flow in NiFi?

2016-06-10 Thread Huagen peng
Hi, I would like to learn about some better practices on logging. Here is what I would imagine in an ideal log for a flow like fetching files from SFTP, processing the files in certain way, and then saving the file to the disk. In the log, I would see that the SFTP step is triggered, with

GetMongo NAR with SSL.

2016-06-10 Thread Mike S
Hello! I'm hoping to get some help with the GetMongo NAR. The one packaged in the current release doesn't support SSL authentication to mongo, however I've noted that the git repo code specifically supports that. So, I tried to build it and deploy from within the current 0.6.1 release.

How to call a stored procedure

2016-06-10 Thread ski n
I want to call a stored procedure as follows: BEGIN SP.INSERT_XML(XMLTYPE(${message}); END; I have two questions: 1) I want to use the flowfile content as input parameter. How is this done with the expression language? 2) Are stored procedures supported by a processor (PutSQL or ExecuteSQL)

Re: Custom processor is failing for concurrency

2016-06-10 Thread Matt Foley
Hi Kumiko, since the Azure SDK calls are controlling an external resource (not part of your PutFileAzureDLStore class) in a non-thread-safe way, the safest thing to do is lock on an object of static scope, hence guaranteed to be shared by all threads wherever spawned, rather than "this".