One way we are currently doing this is: Expose limited set templates (those
will work with pre-defined sources and destinations) to users in a custom
portal, ask user specific configurations in the UI (custom) based on template
selection and integrate this UI with NiFi using REST.
Thanks,
Ravi
Hi,
Nifi version 1.5
I am trying to create connection between output port to input port at root RPG.
Below is my json request:
{"revision": {"clientId": "anonymous","version": 0},"component": {"name":
"","source": { "id": "22df69f1-0162-1000--6c01773e", "groupId":
"22df42c1-0162-10
Hi,
I have re-compiled nifi with mapr dependencies as per instructions at
http://hariology.com/integrating-mapr-fs-and-apache-nifi/
Created process flow with ListFile > FetchFile > PutHDFS. As soon as I start
this process group nifi-bootstrap.log is getting filled with
2018-03-21 22:56:26,806
ess the system. This can be easily done by:
sudo -u "kinit -kt /path/to/your/keytab && maprlogin
kerberos"
Cheers
[1]
https://lists.apache.org/thread.html/af9244266e89990618152bb59b5bf95c9a49dc2428ea3fa0e6aaa682@%3Cusers.nifi.apache.org%3E
[2] https://cwiki.apache.org
o Employee
Cc: "users@nifi.apache.org" , "andre-li...@fucs.org"
Subject: Re: PutHDFS with mapr
Ravi,
Have you tried without recompiling MapR but following the second approach I
listed before?
Kind regards
On Mon, Mar 26, 2018 at 1:17 PM, Ravi Papisetti (rpapiset)
mai
access the system. This can be easily done by:
sudo -u "kinit -kt /path/to/your/keytab && maprlogin
kerberos"
Cheers
[1]
https://lists.apache.org/thread.html/af9244266e89990618152bb59b5bf95c9a49dc2428ea3fa0e6aaa682@%3Cusers.nifi.apache.org%3E
[2] https://cwiki.apache.org/confluence/x/z
't have a mapr-client instance handy but my next steps would be ensuring
the list of directory and subdirectories is complete and if not, add individual
JAR files.
It should work.
On Tue, Mar 27, 2018 at 1:56 AM, Ravi Papisetti (rpapiset)
mailto:rpapi...@cisco.com>> wrote:
Hi Andre
In my experience, if you set first processor to run on "Primary Node", then
remaining flow that directly connects to it will run on that node independent
of how sub-sequent processors are configured to run.
If you really want to distribute the flow, after first step, insert a RPG
processors, th
Hi,
Until recently we were using nifi-1.6 nifi-hbase_1_1_2-client-service-nar
compiled with mapr profile. It has been working fine connecting to mapr
database without any issues.
Recently we are upgrading to nifi-1.7, hbase client service is failing to
connect with same mapr distribution.
wrote:
Ravi
it is very tough to read that stack trace as provided but the core
piece of it seems to be that authorization is failing for "/hbase".
Have you confirmed that the proper authorization configuration is
present?
Thanks
On Sun, Nov 25,
worked fine with writing/reading data
from maprdb. Appreciate any insights why these issues are with nifi-1.7.1 code
base.
Thanks,
Ravi Papisetti
On 25/11/18, 7:45 PM, "Ravi Papisetti (rpapiset)" wrote:
Yes, if I change the version on hbase_client_service_1_1_
though, as long as it is documented.
-Bryan
On Thu, Nov 29, 2018 at 4:30 PM Ravi Papisetti (rpapiset)
wrote:
>
> Hi Joe,
>
>
>
> I am able to resolve this issue by changing hbase-client version to
1.1.8-mapr-1703 in
Hi,
Nifi version 1.7
We have a dataflow that would get data from Oracle database and load into hive
tables.
Data flow is something like below:
GenerateTableFetch -> ExecuteSQL > AvrotoJson/ORC (we tried both) > PutHDFS >
ListHDFS> ReplaceTExt (to build load data query form the file) > PutHiveQ
Hi Experts,
Any thoughts on how to make Hive processors (PutHiveStreaming, PutHiveQL etc.,)
to work with Hive 2.1.1 version. In nifi 1.7.x, hive bundle is compiled with
hive 1.2.1 version.
When I tried to compile with hive 1.2.1 (by updating pom file), lot compilation
issues. I tried to fix so
ks in your use-case, then I'd
recommend disabling "Use Avro Logical Types" at ExecuteSQL.
Thanks,
Koji
On Fri, Mar 8, 2019 at 4:48 AM Ravi Papisetti (rpapiset)
wrote:
>
> Hi,
>
>
>
> Nifi version 1.7
>
Hi,
We are planning to use "ExecuteProcess" to run a sqoop script wrapped by shell.
As part of this we want NiFi to use its service principal in secure mode while
submiting executing the script. Otherwise sqoop script is failed to execute
saying "Caused by GSSException: No valid credentials pro
Hi,
Using Apache Nifi 1.1.2 single node machine, fetching files from hdfs using
FetchHDFS and sending to SplitText. I see SplitText is throwing
arrayindexoutofboundexception for CSV files. It works fine for TAB separated
files.
Below are exception details:
ERROR [Timer-Driven Process Thread-5
Hi,
NiFi version: 1.1.2, Linux
Last night nifi servers got restarted and unable to bring nifi service back up.
I saw this thread
https://community.hortonworks.com/questions/75280/why-does-my-minifi-flow-fail-to-run-when-turning-o.html,
but don’t see any workaround for this.
Below are log mess
n server
Hello
Did the partition nifi use get full? We are making the error handling in such
case more resilient as at time it can result in partial writes.
Joe
Thanks
Joe
On Apr 14, 2017 5:06 PM, "Ravi Papisetti (rpapiset)"
mailto:rpapi...@cisco.com>> wrote:
Hi,
NiFi ve
2017 at 10:26 PM, Ravi Papisetti (rpapiset)
wrote:
> No, NiFi partition is not full. I had to delete all repos to get it back
up.
>
>
>
> Thanks,
>
> Ravi Papisetti
>
>
>
> From: Joe Witt
> Reply-To: "u
This is very useful discussion for me as well.
You mean, step:2 should be something like take flow.xml.gz from NiFi_Home/conf
and check-in into git (step3)?
Thanks,
Ravi Papisetti
From: Juan Sequeiros
Reply-To: "users@nifi.apache.org"
Date: Monday, April 17, 2017 at 11:14 AM
To: Richard Hanso
information?
Thanks,
James
On Thu, Apr 13, 2017 at 3:42 PM, Ravi Papisetti (rpapiset)
mailto:rpapi...@cisco.com>> wrote:
Hi,
Using Apache Nifi 1.1.2 single node machine, fetching files from hdfs using
FetchHDFS and sending to SplitText. I see SplitText is throwing
arrayindexoutofboundexcept
Hi,
We are using NiFi 1.3.0.
Trying to build a template that could take data from hbase and push into
relational store (I know this is little crazy requirement☺). Using below flow
to achieve this:
GetHBase > EvaluateJSONPath (to extract “cells” for all column values) >
ReplaceText (To remove
NiFi Version 1.3
Creating DBCPConnection Pool "/nifi-api/process-groups/" + processorGrpId +
"/controller-services" using nifi 1.3 client.
Below is snippet of code updating properties of connection pool service
String apiUrl = apiBaseUrl + "/nifi-api/controller-services/" +
connectionPoolId;
ntent was
then used to issue an update to enable the controller service then it would set
the password to the masked value which would then end up being wrong.
--
Arpit
On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset)
mailto:rpapi...@cisco.com>> wrote:
NiFi Version 1.3
Creating DBCP
controller service then it would set
the password to the masked value which would then end up being wrong.
--
Arpit
On Jul 13, 2017, at 2:20 PM, Ravi Papisetti (rpapiset)
mailto:rpapi...@cisco.com>> wrote:
NiFi Version 1.3
Creating DBCPConnection Pool "/nifi-api/process-groups/"
rvice entity to
the aforementioned controller service dto.
--
Arpit
On Jul 13, 2017, at 3:16 PM, Ravi Papisetti (rpapiset)
mailto:rpapi...@cisco.com>> wrote:
Thank Arpit. You are right. I sent password property as well while updating the
state and it worked.
On approach you sug
Hi,
We are using Apache NiFi 1.3.0
We have a process flow to copy files from NFS to HDFS (with processors
ListFile, FetchFile and PutHDFS)
In the NiFi process flow, ListFile is configured to listen to a directory on
NFS. When a file (ex: x.csv) is being copied from a windows machine to NFS
(w
se use the 'File Filter' property of ListFile to control ignoring
filenames until they no longer end in 'part'.
Thanks
On Wed, Dec 6, 2017 at 2:14 AM, Ravi Papisetti (rpapiset)
wrote:
> Hi,
>
>
>
> We are using Apache
would not.
The pattern for that example would be
start\..+\.(?!part)
The key part of that is the negative lookahead for ensuring it does
not end in part.
Thanks
On Wed, Dec 6, 2017 at 2:29 AM, Ravi Papisetti (rpapiset)
wrote:
> Yeah..that
Hi,
We are creating processgroups to transfer data from File system to HDFS/HBASE
(separate process groups for each destination).
Simple Example: FileSystem to HDFS process group: We have ListFile, FetchFile
and PutHDFS process to transfer data from File system to HDFS. When “ListFile”
process
en distributing the results is
correct. Before sending to HDFS you'll often want/need to merge
content first unless your objects are large enough to begin with.
Thanks
Joe
On Thu, Dec 21, 2017 at 12:17 PM, Ravi Papisetti (rpapiset)
wrote:
> Hi,
&g
Hi,
Our NiFi cluster is impacted with
https://issues.apache.org/jira/browse/NIFI-4092?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=16150041#comment-16150041
At this point we are not be able to upgrade our environment, is there any
workaround here to r
replied to your github/pr comment. You will need to upgrade to get
the fix which has been released or you'll need to create your own
patched build.
Thanks
Joe
On Thu, Jan 25, 2018 at 5:12 PM, Ravi Papisetti (rpapiset)
wrote:
> Hi,
>
>
>
Hi,
We are trying to update "Run Schedule" and "Scheduling Strategy" of a NiFi
Processor using REST API (PUT /processors/{id}). This works for most of the
processors, but when it is updated for GetSFTP, it overwrites password field
that results authfail exception. If I update scheduling informa
Hi,
We just started exploring apache nifi for data onboarding into MapR
distribution. Have configured putHDFS with yarn-site.xml from on local mapr
client where cluster information is provided, configured the "Directory" with
mapr fs directory to write the files, configured nifi to run as user
the Remote Owner and Remote
Groups settings in the processor appropriate for HDFS permissions?
Conrad
From: "Ravi Papisetti (rpapiset)"
mailto:rpapi...@cisco.com>>
Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>"
mailto:users@nifi.apache.org>>
Dat
er Kerberized? What version are you running? Have you
attempted to use the List/GetHDFS processors? Do they also have errors in
reading?
Thanks!
--aldrin
On Mon, Jun 13, 2016 at 5:19 PM, Ravi Papisetti (rpapiset)
mailto:rpapi...@cisco.com>> wrote:
Thanks Conrad for your reply.
Yes, I
mo
On Jun 13, 2016, at 5:57 PM, Bryan Bende
mailto:bbe...@gmail.com>> wrote:
I'm not sure if this would make a difference, but typically the configuration
resources would be the full paths to core-site.xml and hdfs-site.xml. Wondering
if using those instead of yarn-site.xml changes any
building the nifi-hadoop-libraries-nar
with MapR JARs rather than Apache Hadoop JARs:
https://github.com/apache/nifi/pull/475
Regards,
Matt
On Tue, Jun 14, 2016 at 1:36 PM, Ravi Papisetti (rpapiset)
mailto:rpapi...@cisco.com>> wrote:
Hi,
I have configure the way it is mentioned in belo
r-clusters.conf (assuming you haven't played with client
configs)
Also have in mind that mapr-client.md<http://mapr-client.md/> file seems to be
focused on insecure MapR clusters. Are you using insecure (terrible choice),
MapR Hybrid security (not ideal IMHO) or Kerberos (recomme
Hi,
We are trying to find right workflow for ingesting data from a relational
database to csv files on hadoop.
There are few options we could think of:
* Write a sqoop script and wrap under ExecuteProcess
* Stitch a workflow using "QueryDataBaseTable" -> ConvertAVROToJson ->
ReplaceTex
42 matches
Mail list logo
