Re: Consumekafka error I

Hi Faisal, Some observations and next steps. 1. Brian has a great point of using the more appropriate consumer that will leverage the latest API. 2. This still certainly feels like a resource issue -- testing a single Kafka broker first and not growing the cluster seems odd especially

Re: Consumekafka error I

gt;> so i think there is some thing also wrong with the commit configuration. >> There is some other additional property that guess i need to setup on the >> broker side which will make it scalable. But i'm unable to find it. Kindly >> let me know if y

Re: Consumekafka error I

Hi Faisal, How many partitions are there for that TEST_KAFKA_TOPIC topic? On Thu, Jun 14, 2018 at 9:06 PM Faisal Durrani wrote: > Hi Mark, The heap size is set to 4 gb and the time driven threas count is > set to 32 since its a 8 core node( we have a cluster of 4 nifi nodes) . > Despite of all

Re: Row counts - date lake

the golden gate transactions from Kafka which is received > in Nifi through consume kafka processor . Our data flow then reduces the > golden gate json message and sends the data to the target table in Hbase > using the PutHbase Json processor. > > Thanks, > Faisal > > On Mon,

Re: Row counts - date lake

Hi Faisal, There are various ways this can be handled. But this is going to depend on, how are you receiving data from Oracle via Golden Gate. Are you using the HBase Handler, the HDFS Handler, a Flat File, Kafka, or via another means? Thanks, Andrew On Mon, Jun 11, 2018 at 9:43 AM Faisal

Re: Nifi kafka ser/deser problem

ify the sender and > delete this message and its attachments. > > As emails may be altered, Orange is not liable for messages that have > been modified, changed or falsified. > > Thank you. > > > > ____ > ___

Re: Output Port: Port X is invalid because Output Connection for port X is not defined

09 PM, mayank rathi <mayank.ra...@gmail.com> wrote: > Hello Andres, > > Yes. I am getting same error message. I tried on 1.3 and 1.4 version. > > Thanks for prompt response. > > Regards > Mayank > > On Mon, Sep 4, 2017 at 3:53 PM, Andrew Psaltis <psaltis.and...@gm

Re: Output Port: Port X is invalid because Output Connection for port X is not defined

m/psaltis> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Re: Any possibility to run ExecuteProcess like crontab

--- > > Thanks, > > Hollis > > > *From:* Andrew Psaltis <psaltis.and...@gmail.com> > *Date:* 2017-07-12 19:29 > *To:* users <users@nifi.apache.org> > *Subject:* Re: Any possibility to run ExecuteProcess like crontab > > Hollis, > You shou

Re: Spark Streaming with Nifi

> Do you see this approach logical ? > > Thanks > Shashi > > > On Sun, Jun 4, 2017 at 3:10 PM, Andrew Psaltis <psaltis.and...@gmail.com> > wrote: > >> Hi Shashi, >> I'm sure there is a way to make this work. However, my first question is >> why y

Re: Spark Streaming with Nifi

ller > service ? or any better approach for running spark streaming application > with Nifi ? > > Thanks and Regards, > Shashi > > > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: How to wait for multiple processors' flowfile?

pends on the > flowfiles from A and B. > > So my question is is there a way for C to wait for both flowfiles from A > and B > to arrive and then start the work in C? Thanks. > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis> <ht

Re: Processors on the fly for many sensor devices

rom the Apache NiFi Users List mailing list archive at Nabble.com. > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: error with flow.xml.gz

>) and > delete this email from your system. Any views expressed in this message are > those of the individual sender, except where the sender specifically states > them to be the views of Two Degrees Mobile Limited. We do not guarantee > that this material is free from viruses or any other

Re: Processors on the fly for many sensor devices

before submitting the P/R. Let me know if you want me to add in the processing for the demarcator splitting. Thanks, Andrew On Mon, Oct 31, 2016 at 4:37 PM, Andrew Psaltis <psaltis.and...@gmail.com> wrote: > Hi Davy, > Thanks for the P/R - I merged that in just a little bit ago.

Re: Processors on the fly for many sensor devices

nse > received from the tcp stream. I've seen something similar in the ListenTCP > processor. > > Sorry to hear about your loss. My condolences. > > Regards, > Davy > > > > On Tue, Oct 18, 2016 at 10:16 AM, Andrew Psaltis <psaltis.and...@gmail.com > >

Re: Processors on the fly for many sensor devices

> Currently we have ListenTCP which is waiting for incoming connections, and > PutTCP which makes a connection, but only writes data. > > On Wed, Sep 28, 2016 at 5:02 PM, Andrew Psaltis <psaltis.and...@gmail.com> > wrote: > >> Davy, >> It sounds like you nee

Re: putazureeventhub error

Hi Aaron, Attached is a simple flow that publishes events to an eventhub I have -- if you want to try and publish to it, hit me up privately and I can share the info for the hub. Thanks, Andrew On Thu, Aug 11, 2016 at 9:11 AM, Andrew Psaltis <psaltis.and...@gmail.com> wrote: >

Re: [ANNOUNCE] Apache NiFi 1.0.0-Beta release

sion=12338066 > Release note highlights can be found here:https://cwiki.apache.org/ > confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.0.0-Beta > Thank youThe Apache NiFi team > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis>

Re: putazureeventhub error

rocessorNode.access$ > 100(StandardProcessorNode.java:90) ~[na:na] > > at org.apache.nifi.controller.StandardProcessorNode$1.run( > StandardProcessorNode.java:1230) ~[na:na] > > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >

Re: Logstash/ Filebeat/ Lumberjack -> Nifi

y prices quoted are only valid if followed up by a formal written quote. >> >> SecureData Europe Limited. Registered in England & Wales 04365896. >> Registered Address: SecureData House, Hermitage Court, Hermitage Lane, >> Maidstone, Kent, ME16 9NT >> > > > > ***This email originated outside SecureData*** > > Click here > <https://www.mailcontrol.com/sr/CdEQiWndhxLGX2PQPOmvUsrLibhXE7+SpVooqDfjfmrv9UcAoCvw58JRjsQQpswieUDNxz32L0IKghm6!2a+jw==> > to report this email as spam. > -- Thanks, Andrew Subscribe to my book: Streaming Data <http://manning.com/psaltis> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>

Re: Help Wanted

iew. > > On Tue, May 3, 2016 at 11:29 AM, Andrew Psaltis <psaltis.and...@gmail.com> > wrote: > >> Totally agree on all fronts. Would seem like it makes sense for a >> documentation PR to be opened soon with updates to the >> https://cwiki.apache.org/confluence/dis

Re: Help Wanted

t;> specifically due to the inline review/comment capabilities provided by >>> GitHub. >>> >>> Cheers >>> Oleg >>> >>> > On May 3, 2016, at 11:18 AM, Andrew Psaltis <psaltis.and...@gmail.com> >>> wrote: >>> >

Help Wanted

Oleg, I would love to help -- couple of quick questions: The GH PR's are ~60 as you indicated, but the How To Contribute guide (Code review process -- https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-CodeReviewProcess ) shows a JIRA list with patches available.