NiFi 1.3 - PublishKafka_0_10 - A message in the stream exceeds the maximum allowed message size of 1048576 bytes

2017-11-17 Thread mayank rathi
Hello All, I am getting this error in PublishKafka_0_10 processor for a message of size 2.08 MB. I have updated Max Request Size to 10 MB in processor properties and max.request.size to 10 MB in Kafka's server.properties. After reboot Kafka Broker I can see that max.request.size = 10 MB in Kafka l

Re: NiFi: Extracting text from images

2017-10-12 Thread mayank rathi
setup. > > Thanks > Jeremy Dyer > > On Sun, Oct 8, 2017 at 9:53 AM, mayank rathi > wrote: > >> Thanks Joe and Kevin for replying. >> >> Just to close the loop on this issue. I was "incorrectly" expecting >> ExtractImageMetadata to identi

PutSQL: java.sql.BatchUpdateException: invalid batch command

2017-10-12 Thread mayank rathi
Hello All, Can someone please help in understanding what I am doing wrong here? I am passing following values to PutSQL processor sql.args.1.type 2005 sql.args.1.value V99|1.0| E99|BILLING|1234567|11049532|350|201706|201709| R03|350|11049532|4| |KK|174|A|07/27/1948| |322| |0|F| | |201712|201709

Re: NiFi: Extracting text from images

2017-10-08 Thread mayank rathi
en you tried it what happened? Did you look at the flow file > attributes after the extraction? > > Can you share a jpeg you're using that is not working as you'd expect? > > Thanks > Joe > > On Thu, Sep 28, 2017 at 4:12 PM, mayank rathi &

NiFi: Extracting text from images

2017-09-28 Thread mayank rathi
Hello All, Can NiFi extract text from jpg images? If Yes, then which processor do I need to use? I tried ExtractImageMetadata processor and it did not help. Thanks in advance.

Re: How to get RouteOnContent to work

2017-09-07 Thread mayank rathi
Hello James, I am using RouteOnContent with following properties and it's working fine for me: Match Requirement: content must contain match Character Set: UTF-8 Content Buffer Size: 1 MB ExpectedEvent: BILLING|PAYMNT|PAYADJ|CALLADJ Basically I am looking for records that contain any of the foll

Re: PutSQL error: java.lang.String cannot be cast to oracle.sql.CLOB

2017-09-05 Thread mayank rathi
Hello All, Logged https://issues.apache.org/jira/browse/NIFI-4352 for this issue. Thanks!! On Mon, Sep 4, 2017 at 10:24 AM, mayank rathi wrote: > Hello All, > > I am passing sql.args.1.type = 2005 and sql.args.1.value as a CLOB value > to PutSQL and it is throwing below err

Re: PutSQL error: java.lang.String cannot be cast to oracle.sql.CLOB

2017-09-05 Thread mayank rathi
Any thoughts on this error? On Mon, Sep 4, 2017 at 10:24 AM, mayank rathi wrote: > Hello All, > > I am passing sql.args.1.type = 2005 and sql.args.1.value as a CLOB value > to PutSQL and it is throwing below error. How do I resolve this error? > > 2017-09-04

Re: Output Port: Port X is invalid because Output Connection for port X is not defined

2017-09-04 Thread mayank rathi
the objective of an Output Port is > to transfer data out of a Process Group and unless that is configured, then > NiFi will let you know that the Output Port is currently in an invalid > state. > > Does that help? > > Thanks, > Andrew > > On Mon, Sep 4, 2017 at 4:09 P

Re: Output Port: Port X is invalid because Output Connection for port X is not defined

2017-09-04 Thread mayank rathi
gt; > [image: Inline image 2] > > Also, what version of NiFi are you using? > > Thanks, > Andrew > > On Mon, Sep 4, 2017 at 3:37 PM, mayank rathi > wrote: > >> Hello All, >> >> I dragged an Output Port in canvas inside a Process Group and I am >> ge

Output Port: Port X is invalid because Output Connection for port X is not defined

2017-09-04 Thread mayank rathi
Hello All, I dragged an Output Port in canvas inside a Process Group and I am getting error that - Port X is invalid because Output Connection for port X is not defined. What configuration am I missing? Thanks!!

PutSQL error: java.lang.String cannot be cast to oracle.sql.CLOB

2017-09-04 Thread mayank rathi
Hello All, I am passing sql.args.1.type = 2005 and sql.args.1.value as a CLOB value to PutSQL and it is throwing below error. How do I resolve this error? 2017-09-04 10:17:24,924 INFO [StandardProcessScheduler Thread-5] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled PutSQL[id=69534f73-821e-1782-f

CLOB in sql.args.N.value

2017-09-04 Thread mayank rathi
Hello All, Can I pass a CLOB as sql.args.N.value? Are their any size limitations for attributes? Thanks!!

PutSQL, Select query and CLOB

2017-09-03 Thread mayank rathi
Hello All, I want to run below query in PutSQL. This query is calling an Oracle function and accepts a CLOB value as parameter. select *obds_loader*.getmain(:clob)from dual Does PutSQL support SELECT queries? Can I pass a CLOB as attribute in NiFi? If Yes, then what would be the value for sql.ar

Re: Multiple or statements

2017-08-28 Thread mayank rathi
Hello Laurens, eventName is an attribute of flow file or is part of content ( data) ? If it's content then please use RouteOnContent processor On Mon, Aug 28, 2017 at 10:54 AM, Laurens Vets wrote: > Hi Mark, > > Can it be that there's something wrong with the below expression? > > Basically, I'

Re: Missing nifi-app.log files

2017-08-17 Thread mayank rathi
I had same observation. I was trying to collect concise logs to open a JIRA and deleted all the existing logs but NiFi did not generate new logs after cleanup. I had to restart NiFi to generate new logs. On Thu, Aug 17, 2017 at 4:57 PM, Russell Bateman wrote: > James, > > It's the case that, NiF

Re: Pivot: Columns to Rows

2017-08-10 Thread mayank rathi
koverflow.com/a/42639228/70465 > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On Aug 10, 2017, at 9:43 AM, mayank rathi wrote: > > Hello All, > > How can I pivot record

Pivot: Columns to Rows

2017-08-10 Thread mayank rathi
Hello All, How can I pivot record based data from columns to rows. Here is an example of what I want to achieve: Input: A,B,C M,N,O Output: A,B A,C M,N M,O Thanks!! -- NOTICE: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileg

Re: Cluster is unable to service request to change flow

2017-08-07 Thread mayank rathi
ely remove the node from the cluster > or reconnect it. > > Hope this helps, > Pierre > > 2017-08-07 18:59 GMT+02:00 mayank rathi : > >> Hello All, >> >> I was troubleshooting an issue with my cluster ( 3 node) and I ran >> bin/nifi.sh stop on the primar

Cluster is unable to service request to change flow

2017-08-07 Thread mayank rathi
Hello All, I was troubleshooting an issue with my cluster ( 3 node) and I ran bin/nifi.sh stop on the primary node. Controller chose another node as Primary and cluster is processing data but when I try to stop a Processor Group in NiFi UI, I get below error. "Cluster is unable to service request

How to send data stored in Kafka from beginning to a new consumer using NiFi?

2017-08-07 Thread mayank rathi
Hello All, How can I add a new NiFi consumer to a Kafka topic and send data from beginning? I do not see any setting in ConsumerKafkaRecord_0_10 processor to accomplish this. I am looking for functionality similar to --from-beginning parameter in Kafka Console Consumer. "./bin/kafka-console-cons

How to add headers to CSV file

2017-08-02 Thread mayank rathi
Hello All, I am working on a NiFi flow to read data from a database and dump it into a CSV file. I will use ExecuteSQL to read data and PutFile to generate CSV file. Which processor can I use to add header to this CSV file? Thanks in advance MR -- NOTICE: This email message is for the sole us