SelectHiveQL Incremental

Hi all, The GenerateTableFetch processor allow defining a < max value column > to get only recent rows (for example). Is there any way of doing the same with the SelectHiveQL ? Currently, it seems the SelectHiveQL processor always gets all the rows each time it is run while I would like to get

RE: SFTP and Data Integrity

here - https://issues.apache.org/jira/browse/NIFI Cheers Oleg On Jan 5, 2017, at 4:23 AM, Provenzano Nicolas mailto:nicolas.provenz...@gfi.fr>> wrote: Hi all, Is there any way of configuring the fetchSFTP processor to calculate a remote file checksum in order to check data integrity ? If

SFTP and Data Integrity

Hi all, Is there any way of configuring the fetchSFTP processor to calculate a remote file checksum in order to check data integrity ? If there isn't with this processor, is there any other ways of doing it ? Thanks in advance, BR Nicolas Provenzano

Nifi "Hive" with CDH

Hi all, We are currently working on NiFi Hive processors (selectHiveQL, putHiveStreaming...). We are testing them with a CDH platform (and with a "Apache" platform). Hive 1.0.0 is deployed with the last releases of CDH (5.9) while Nifi is built with Hive 1.2.1 libraries. We then have serious

PutHiveStreaming when metastore is in embedded mode

Hi all, Can the PutHiveStreaming processor work when the Hive Metastore server is deployed in embedded mode ? I tried to configure the processor to connect to a such configured metastore server and it fails to connect. Do I need to deploy the metastore server in remote mode ? (I wanted to use

putHDFS/getHDFS/fetchHDFS : HDFS service name instead of master node host name.

Hello all, I configured a putHDFS processor as follows : Hadoop Configuration Resources : /etc/hadoop/conf/core-site.xml,/etc/hadoop/conf/hdfs-site.xml The core site file contains : fs.defaultFS hdfs://mas:8020/ While the hdfs site file contains : dfs.nameservices m

Merge Content : triggering merge according to a field

Hi all, Yet another question... I defined a flow to process session information coming from CSV files. Each record contains a session ID, a session state and some session counters. The merge content processor allows merging flowfiles according to an attribute (the correlation attribute name) s

Load distribution on a Nifi cluster

Hi all, As described in many topics, the processing of collected files can be distributed over a Nifi cluster by using a remote process group. For example, we can use a listSFTP processor, connect it to a port that is the input of a remote process group. However, when configuring the URL of th

Flow file replication

Hi all, Is there a way to replicate the repositories in order to be able to resume the processing of data when a nodes crashes ? What is the recommendation to avoid losing data in case a node crashes ? Thanks in advance BR Nicolas - Gfi Informatique Telecom Nicolas Provenzano BU Telecom / Cu

RE: Nifi vs Sqoop

d.GenerateTableFetch/index.html [2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.QueryDatabaseTable/index.html On Wed, Nov 9, 2016 at 4:37 AM, Provenzano Nicolas mailto:nicolas.provenz...@gfi.fr>> wrote: Hi all, I have the following requirements :

RE: Nifi vs Sqoop

ur data. Regards, Matt [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.GenerateTableFetch/index.html On Wed, Nov 9, 2016 at 4:37 AM, Provenzano Nicolas wrote: > Hi all, > > > > I have the following requirements : > > &g

Nifi vs Sqoop

Hi all, I have the following requirements : * I need to load at day 1 a full SQL table, * And then need to incrementally load new data (using capture data change mechanism). Initially, I was thinking using Sqoop to do it. Looking at Nifi and especially the QueryDatabaseTable

ListSFTP processor with dynamic hostname

Hi all, We need to collect files using SFTP from several data sources. We can't add one ListSFTP processor per data source since, first, it would require define many and it's not dynamic at all. Our initial idea was to add processors to read a configuration file (containing a list of hostnames

RE: Access denied for kerberos users

What policies did you grant to the kerberos user in NiFi? At a minimum they need a policy for "view the user interface" from the global policies in the top-right menu. -Bryan On Mon, Sep 26, 2016 at 11:43 AM, Provenzano Nicolas mailto:nicolas.provenz...@gfi.fr>> wrote: Hi all,

RE: Access denied for kerberos users

olicies did you grant to the kerberos user in NiFi? At a minimum they need a policy for "view the user interface" from the global policies in the top-right menu. -Bryan On Mon, Sep 26, 2016 at 11:43 AM, Provenzano Nicolas mailto:nicolas.provenz...@gfi.fr>> wrote: Hi all, I co

Access denied for kerberos users

Hi all, I configured an 1.0.0 NIFI instance to use Kerberos services for authentication. I can connect to the UI using the certificate corresponding to the user declared in the Initial Admin Identity. However, when I try to connect using a user declared in the Kerberos server : 1. Based

RE: Nifi UI configured to be accessed over HTTPS not displayed in Internet Explorer

ou can start installing Edge, but I don’t believe it’s available on Win7. Thanks, Peter From: Provenzano Nicolas [mailto:nicolas.provenz...@gfi.fr<mailto:nicolas.provenz...@gfi.fr>] Sent: Thursday, September 22, 2016 1:25 AM To: users@nifi.apache.org<mailto:users@nifi.apache.

Nifi UI configured to be accessed over HTTPS not displayed in Internet Explorer

Hi all, I installed a Nifi 1.0.0 instance and configured the User Interface to be accessed over HTTPS. I installed the client certificate in IE and Firefox. Everything works fine with Firefox when connecting to https://host1:9443/nifi/. When I try to connect using IE, the tab is correctly renam