Re: Convert Avro to ORC or JSON processor - retaining the data type

ks in your use-case, then I'd recommend disabling "Use Avro Logical Types" at ExecuteSQL. Thanks, Koji On Fri, Mar 8, 2019 at 4:48 AM Ravi Papisetti (rpapiset) wrote: > > Hi, > > > > Nifi version 1.7 > &g

Hive processors to make it compatible with Hive 2.1.1

Hi Experts, Any thoughts on how to make Hive processors (PutHiveStreaming, PutHiveQL etc.,) to work with Hive 2.1.1 version. In nifi 1.7.x, hive bundle is compiled with hive 1.2.1 version. When I tried to compile with hive 1.2.1 (by updating pom file), lot compilation issues. I tried to fix

Convert Avro to ORC or JSON processor - retaining the data type

Hi, Nifi version 1.7 We have a dataflow that would get data from Oracle database and load into hive tables. Data flow is something like below: GenerateTableFetch -> ExecuteSQL > AvrotoJson/ORC (we tried both) > PutHDFS > ListHDFS> ReplaceTExt (to build load data query form the file) >

Re: hbaseclient service is failed to enable

h, as long as it is documented. -Bryan On Thu, Nov 29, 2018 at 4:30 PM Ravi Papisetti (rpapiset) wrote: > > Hi Joe, > > > > I am able to resolve this issue by changing hbase-client version to 1.1.8-mapr-1703 in the nifi-hbase_

Re: hbaseclient service is failed to enable

, but this worked fine with writing/reading data from maprdb. Appreciate any insights why these issues are with nifi-1.7.1 code base. Thanks, Ravi Papisetti On 25/11/18, 7:45 PM, "Ravi Papisetti (rpapiset)" wrote: Yes, if I change the version on hbase_client_service_1_1_

Re: hbaseclient service is failed to enable

Ravi it is very tough to read that stack trace as provided but the core piece of it seems to be that authorization is failing for "/hbase". Have you confirmed that the proper authorization configuration is present? Thanks On Sun, Nov 25, 2018 at 4:50 PM Ravi

hbaseclient service is failed to enable

Hi, Until recently we were using nifi-1.6 nifi-hbase_1_1_2-client-service-nar compiled with mapr profile. It has been working fine connecting to mapr database without any issues. Recently we are upgrading to nifi-1.7, hbase client service is failing to connect with same mapr distribution.

Re: Clustered Flow Execution Help

In my experience, if you set first processor to run on "Primary Node", then remaining flow that directly connects to it will run on that node independent of how sub-sequent processors are configured to run. If you really want to distribute the flow, after first step, insert a RPG processors,

Re: PutHDFS with mapr

u "kinit -kt /path/to/your/keytab && maprlogin kerberos" Cheers [1] https://lists.apache.org/thread.html/af9244266e89990618152bb59b5bf95c9a49dc2428ea3fa0e6aaa682@%3Cusers.nifi.apache.org%3E [2] https://cwiki.apache.org/confluence/x/zI5zAw On Fri, Mar 23, 2018 at 5:05 AM, Ra

Re: PutHDFS with mapr

Cheers [1] https://lists.apache.org/thread.html/af9244266e89990618152bb59b5bf95c9a49dc2428ea3fa0e6aaa682@%3Cusers.nifi.apache.org%3E [2] https://cwiki.apache.org/confluence/x/zI5zAw On Fri, Mar 23, 2018 at 5:05 AM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com<mailto:rpapi...@cisco.com>> wro

Re: PutHDFS with mapr

ling MapR but following the second approach I listed before? Kind regards On Mon, Mar 26, 2018 at 1:17 PM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com<mailto:rpapi...@cisco.com>> wrote: Thanks Andre. In the absence of FUSE license, I have recompiled NiFi with mpar dependencies. I sta

Re: PutHDFS with mapr

Cheers [1] https://lists.apache.org/thread.html/af9244266e89990618152bb59b5bf95c9a49dc2428ea3fa0e6aaa682@%3Cusers.nifi.apache.org%3E [2] https://cwiki.apache.org/confluence/x/zI5zAw On Fri, Mar 23, 2018 at 5:05 AM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com<mailto:rpapi...@cisco.com>

PutHDFS with mapr

Hi, I have re-compiled nifi with mapr dependencies as per instructions at http://hariology.com/integrating-mapr-fs-and-apache-nifi/ Created process flow with ListFile > FetchFile > PutHDFS. As soon as I start this process group nifi-bootstrap.log is getting filled with 2018-03-21 22:56:26,806

Connection between RPG and input port is failing via API

Hi, Nifi version 1.5 I am trying to create connection between output port to input port at root RPG. Below is my json request: {"revision": {"clientId": "anonymous","version": 0},"component": {"name": "","source": { "id": "22df69f1-0162-1000--6c01773e", "groupId":

Re: Best Pratices for implementing a source catalog for Nifi

One way we are currently doing this is: Expose limited set templates (those will work with pre-defined sources and destinations) to users in a custom portal, ask user specific configurations in the UI (custom) based on template selection and integrate this UI with NiFi using REST. Thanks, Ravi

Re: NiFi node is switching between "Connecting" and "Connected" state - NIFI-4092 issue

wrote: Ravi, I replied to your github/pr comment. You will need to upgrade to get the fix which has been released or you'll need to create your own patched build. Thanks Joe On Thu, Jan 25, 2018 at 5:12 PM, Ravi Papisetti (rpapiset) <rpapi...@cisco.c

NiFi node is switching between "Connecting" and "Connected" state - NIFI-4092 issue

Hi, Our NiFi cluster is impacted with https://issues.apache.org/jira/browse/NIFI-4092?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel=16150041#comment-16150041 At this point we are not be able to upgrade our environment, is there any workaround here to resolve this

Clarification on load distribution on NiFi cluster

Hi, We are creating processgroups to transfer data from File system to HDFS/HBASE (separate process groups for each destination). Simple Example: FileSystem to HDFS process group: We have ListFile, FetchFile and PutHDFS process to transfer data from File system to HDFS. When “ListFile”

Re: NiFi copying windows .part files

iddle.part would not. The pattern for that example would be start\..+\.(?!part) The key part of that is the negative lookahead for ensuring it does not end in part. Thanks On Wed, Dec 6, 2017 at 2:29 AM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com

NiFi copying windows .part files

Hi, We are using Apache NiFi 1.3.0 We have a process flow to copy files from NFS to HDFS (with processors ListFile, FetchFile and PutHDFS) In the NiFi process flow, ListFile is configured to listen to a directory on NFS. When a file (ex: x.csv) is being copied from a windows machine to NFS

Re: Failed to create dbcp connection pool - using nifi API

erServiceDTO and set its id and state and then set the component of the new controller service entity to the aforementioned controller service dto. -- Arpit On Jul 13, 2017, at 3:16 PM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com<mailto:rpapi...@cisco.com>> wrote: Thank Arpit. You

Re: Failed to create dbcp connection pool - using nifi API

nd pass that to the put request to set the required state. -- Arpit On Jul 13, 2017, at 3:02 PM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com<mailto:rpapi...@cisco.com>> wrote: Controller service is instantiated from template. After instantiation, controller service config is up

Re: Failed to create dbcp connection pool - using nifi API

when you create a controller service returns a masked password. So if this response content was then used to issue an update to enable the controller service then it would set the password to the masked value which would then end up being wrong. -- Arpit On Jul 13, 2017, at 2:20 PM, Ravi Pap

Failed to create dbcp connection pool - using nifi API

NiFi Version 1.3 Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId + "/controller-services" using nifi 1.3 client. Below is snippet of code updating properties of connection pool service String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" + connectionPoolId;

Hbase to Relational

Hi, We are using NiFi 1.3.0. Trying to build a template that could take data from hbase and push into relational store (I know this is little crazy requirement☺). Using below flow to achieve this: GetHBase > EvaluateJSONPath (to extract “cells” for all column values) > ReplaceText (To remove

Re: SplitText is throwing ArrayIndexOutofBoundException

ple of the data you are splitting and the settings of the SplitText processor? Is there more error stack trace information? Thanks, James On Thu, Apr 13, 2017 at 3:42 PM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com<mailto:rpapi...@cisco.com>> wrote: Hi, Using Apache Nifi 1.1.2

Re: Some question

This is very useful discussion for me as well. You mean, step:2 should be something like take flow.xml.gz from NiFi_Home/conf and check-in into git (step3)? Thanks, Ravi Papisetti From: Juan Sequeiros Reply-To: "users@nifi.apache.org" Date: Monday,

Re: NiFi failed to restart after power outage on server

On Sun, Apr 16, 2017 at 10:26 PM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com> wrote: > No, NiFi partition is not full. I had to delete all repos to get it back up. > > > > Thanks, > > Ravi Papisetti > > >

Re: NiFi failed to restart after power outage on server

<users@nifi.apache.org> Subject: Re: NiFi failed to restart after power outage on server Hello Did the partition nifi use get full? We are making the error handling in such case more resilient as at time it can result in partial writes. Joe Thanks Joe On Apr 14, 2017 5:06 PM, "Ravi

NiFi failed to restart after power outage on server

Hi, NiFi version: 1.1.2, Linux Last night nifi servers got restarted and unable to bring nifi service back up. I saw this thread https://community.hortonworks.com/questions/75280/why-does-my-minifi-flow-fail-to-run-when-turning-o.html, but don’t see any workaround for this. Below are log

SplitText is throwing ArrayIndexOutofBoundException

Hi, Using Apache Nifi 1.1.2 single node machine, fetching files from hdfs using FetchHDFS and sending to SplitText. I see SplitText is throwing arrayindexoutofboundexception for CSV files. It works fine for TAB separated files. Below are exception details: ERROR [Timer-Driven Process

Can we configure NiFi to run execute process with specific Kerberos Principal?

Hi, We are planning to use "ExecuteProcess" to run a sqoop script wrapped by shell. As part of this we want NiFi to use its service principal in secure mode while submiting executing the script. Otherwise sqoop script is failed to execute saying "Caused by GSSException: No valid credentials

Re: Writing files to MapR File system using putHDFS

Are you using insecure (terrible choice), MapR Hybrid security (not ideal IMHO) or Kerberos (recommended IMHO)? Cheers On Wed, Jun 15, 2016 at 3:36 AM, Ravi Papisetti (rpapiset) <rpapi...@cisco.com<mailto:rpapi...@cisco.com>> wrote: Hi, I have configure the way it is mentioned in belo

Re: Writing files to MapR File system using putHDFS

the latter, is security established for your ZooKeeper ensemble? Is your target cluster Kerberized? What version are you running? Have you attempted to use the List/GetHDFS processors? Do they also have errors in reading? Thanks! --aldrin On Mon, Jun 13, 2016 at 5:19 PM, Ravi Papisetti (rpap