Re: Connecting to OPCUA

Thank you for pointing this out, wade and Aldrin. i will set it up correctly. I have a follow up question to this. Currently the plan is to use the GetValue processor to continuously receive values in a raspberry pi that will in turn push the values to a NiFi server installed in cloud. We want

Re: Security between MiNiFi and NiFi

Hi Varsha, The preferred mechanism for accomplishing this is via Site to Site and Remote Processing Groups [1]. This makes use of the configured Security Properties [2]. For other means of data egress, these would vary depending on specific processor or extension used. [1]

Re: List processors

Ah I see, the providers say which scopes they support, and then the framework only lets them be used for the appropriate scope, which makes sense. I suppose the WriteAheadLocalStateProvider could be modified to support CLUSTER scope, although this seems like a bad idea to give people an option

Re: List processors

Thanks Bryan just tried and NIFI does not start because of this: " Cannot use Cluster State Provider ( WriteAheadLocalStateProvider ) as it only supports scope(s) {LOCAL} but instance is configured to use scope CLUSTER" On Fri, Apr 21, 2017 at 2:12 PM Bryan Bende wrote: >

Re: List processors

Juan, I believe from the processor side of the things, when a processor calls save/retrieve on the state manager, the processor has to specify a context like CLUSTER or LOCAL. If you specify CLUSTER, and no clustered state provider exists, then it will save it to the local provider. This allows a

Re: List processors

To add more to the issue I see this on my log: "Failed to restore processor state; yielding java.io.IOException; Failed to obtain value from Zookeeper for component " with exception code CONNECTIONLOSS So this is confirming what I expected and at this point not sure if this is a bug or

Security between MiNiFi and NiFi

Hello, Can you refer me to the documentation for setting up a secure communication between MiNiFi (on an IoT device) and NiFi on EC2 instance? Thanks, Varsha Registered Office: 130 Pandurang Budhkar Marg, Worli, Mumbai - 400018; Corporate Identity number: L28920MH1957PLC010839; Tel.: +91 (22)

Re: How to get ftp file according to Current date?

You could use the combination of ListFTP and FetchFTP (this is, most of the time, a better approach), and between the two processors you could do a RouteOnAttribute and only keep the flow files with the filename you are looking for. Pierre. 2017-04-21 13:29 GMT+02:00 prabhu Mahendran

List processors

Hello all, My preliminary testing shows that if I run ListS3 ( maybe all list processors? ) processor on a cluster and that cluster is not running or configured to talk to zookeeper that he does not maintain state at all even though I would expect him to maintain state locally. EX:

Re: Help with SFTP processor

Joe - Still working with the SFTP chain. We have been testing and have encountered a different problem. During some transfers but not all, when doing a transfer of a large number of files (>10), some of the files are not transferring. I'm getting no errors at all. Here is the chain I'm using:

Re: NiFi best practices to manage big flowfiles

Simone, There is a Feature Proposal that was put together on our wiki at [1] that proposes a way to have a FlowFile refer to content that lives elsewhere outside of the content repo itself. I think this is what you're getting at. It's a great idea, but I don't know that any progress has yet

Re: NiFi best practices to manage big flowfiles

Let me ask you this. All those processing cli steps, do they change file format, content, etc? If yes, NiFi is not doing anything that you aren't doing already. E.g. unpacking a file requires space for the original and decompressed file to be available. You can use ListFile and not move any files

How to get ftp file according to Current date?

I have tried that "GetFTP" processor in which downloads file from FTP accoding to the two attributes 1."FileFilterRegex" -Name of file in FTP 2."RemotePath"-Path of an FTP file. I wants to download the File from FTP Server only if it having today's date which is append with filename. *For

Re: NiFi best practices to manage big flowfiles

Dear Andrew, I am working with Damiano on this, so let me first thank you for your indications. The use case is as follows: - a satellite acquisition is placed on a shared file system. It can be significative in size, e.g. 10GB - it has to be pulled through a chain of operations out of a larger

Re: NiFi best practices to manage big flowfiles

Hi, First, there won't be multiple copies of a file within NiFi. If you pass around the content and don't change it (only attributes), it will merely point a reference to it, no more. You need to decide if you want to delete processed files, this is what GetFile does. Might want to look into

Re: Clustering Best Practices?

Indeed. A very important distinction. Thank you for the correction Andrew. I'll be more careful with my terminology. -Jim On Thu, Apr 20, 2017 at 6:40 PM, Andrew Grande wrote: > BTW, your NiFi instance is not single-threaded, it's a single node. It > still runs multiple

NiFi best practices to manage big flowfiles

Hi list, I'm a NiFi newbie and I'm trying to figure out the best way to use it as a batch ingestion system for satellite imagery as raster files.The files are pushed on the FS by an external system and then they must be processed and published through WMS protocols.I tried to draft the flow