Re: run.as option does not work other than Nifi user

are you running from RPM? If yes, the permissions of the /opt/nifi/ are quite restrictive. Cheers On Fri, Jun 3, 2016 at 4:39 PM, Shashi Vishwakarma wrote: > Hi > > I want to run my NiFi application using ec2-user rather than default nifi > user. I changed *run.as =ec2-user* in

Re: Merge multiple flowfiles

Huge Just to close the loop on this one, I also wanted to point out this JIRA https://issues.apache.org/jira/browse/NIFI-1926 for general purpose aggregation processor which indeed would support multiple connections, configurable aggregation, release and correlation strategies. It would be nice

Re: Merge multiple flowfiles

Huagen, I also want to apologize for my spell-checker butchering your name ;) Cheers Oleg On Jun 3, 2016, at 8:03 AM, Oleg Zhurakousky mailto:ozhurakou...@hortonworks.com>> wrote: Huge Just to close the loop on this one, I also wanted to point out this JIRA https://issues.apache.org/jira/brow

How does GetSFTP treat files being loaded?

Hi, I need to get files from a SFTP server and then remove the files afterward. GetSFTP seems to be the processor to use. If a user uploads a large file, say 20G, to the server and the GetSFTP processor happens to be running in the middle of the uploading, what is the expected behavior. Does

Re: How does GetSFTP treat files being loaded?

Adding to what Oleg said... GetSFTP has a property called Ignore Dotted Files which defaults to true and tells GetSFTP to ignore filenames that begin with dots which can help in this scenario if the uploader can upload to temporary file starting with a dot and rename when done. -Bryan On Fri, J

Re: How does GetSFTP treat files being loaded?

Huagen Just to clarify. There isn’t really an SFTP server. SFTP is just an FTP layer over SSH - allowing you to get FTP-like experience when accessing file system over SSH. Now, tis your question. I am assuming you are implying that some other system is uploading a large file. In any event a ty

Piping commands in ExecuteStreamCommand/ExecuteProcess

Hi, I was trying to get the ExecuteStreamCommand processor to execute one command and then pipe the result right to another Linux command in the same processor. This is how I try to configure the processor: Property Value Command Arguments -r;${log_dir};|;b

Re: Piping commands in ExecuteStreamCommand/ExecuteProcess

Hi Huagen, It looks to me like the ExecuteStreamCommand is designed to run a single command. A potential work around could be to make a shell script that executes the pipeline you’d like and then use the ExecuteStreamCommand to invoke that. I’d use your shell command (e.g. /bin/bash) for the c

Re: Piping commands in ExecuteStreamCommand/ExecuteProcess

Bryan, That was a brilliant suggestion. I got it to work. Many thanks. Huagen > 在 2016年6月3日，下午12:51，Bryan Rosander 写道： > > Hi Huagen, > > It looks to me like the ExecuteStreamCommand is designed to run a single > command. A potential work around could be to make a shell script that > ex

Custom processor is failing for the custom processor

Hello, I wrote the following custom service control and processor. When the custom processor is running concurrently, it's failing often with several different errors. Are there any special handlings for concurrently that I need to add in the custom processor? I wrote the sample Java program

Re: Custom processor is failing for the custom processor

Hello, Would you be able to share your code for PutFileAzureDLStore so we can help identify if there is a concurrency problem? -Bryan On Fri, Jun 3, 2016 at 3:39 PM, Kumiko Yada wrote: > Hello, > > > > I wrote the following custom service control and processor. When the > custom processor is

RE: Custom processor is failing for concurrency

Here is the code, https://github.com/kyada1/PutFileAzureDLStore. Thanks Kumiko From: Bryan Bende [mailto:bbe...@gmail.com] Sent: Friday, June 3, 2016 12:57 PM To: users@nifi.apache.org Subject: Re: Custom processor is failing for the custom processor Hello, Would you be able to share your code

Re: Custom processor is failing for concurrency

Kumiko It appears that the current state of the source you linked in is not in sync with what is in the stack trace. Perhaps you have made some code modifications (e.g., line 218 is an empty line in code while it has a pointer in the star trace). In any event, from what I can see the error is c

Re: Custom processor is failing for concurrency

It is hard to say for sure, but I think your NiFi processor is generally ok regarding thread safety, but I think there could be a problem in the Azure SDK code... RequestFactory has an instance of BaseUrl and every time RequestFactory.create() is called, it calls BaseUrl.url(). The implementation

RE: Custom processor is failing for concurrency

Here is the callstacks with the latest changes. 2016-06-03 14:32:32,766 WARN [Timer-Driven Process Thread-4] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutFileAzureDLStore[id=2133fdad-54b7-46a5-9bcf-958ddd8cfac0] due to uncaught Exception: com.microsoft.rest.ServiceExceptio

Processor Question

Hello:     I would like to take a file and remove all but the first 10 lines of or the first 256 bytes. Is there a way to do this with one of the existing processors?     Second question     Is there a processor that would remove say everything from