Hi Team,
Is there a way we can read provenance data from files stored in disk.
We are facing issue with not able to search on provenance from UI. It is
stuck on searching screen. Looking for a way to extract information from
file.
Thanks Joe.
erformance (throughput) and memory
> utilization.
>
> But ideally you want to use the version that matches your Kafka brokers.
> So if you’re using Kafka 2.6 you should use ConsumeKafka_2_6. If you are
> using Kafka 2.0 - 2.5 you should use ConsumeKafka_2_5 and so on.
>
> Thanks
&g
Hi Team,
I remember seeing a Kafka consumer named ConsumeKafka in version lower than
1.15 , do we have replacement for that processor in 1.15
--
Regards,
Midhun Mohan
Hi all,
I am looking for options on how to secure parameters passed in the header
for this processor, looks like it currently has it as plain text in the
processor. Whoever has access to the system can grab and use it.
How can I restrict this?
--
Regards,
Midhun Mohan
Hi Team,
When I tried setting up a secure cluster, I am getting following error
*[Replicate Request Thread-3] o.a.n.c.c.h.r.ThreadPoolRequestReplicator
Failed to replicate request GET /nifi-api/flow/current-user*
*Please help me to figure this out.*
--
Regards,
Midhun Mohan
Awesome Video, thank you. That really helps
On Mon, Sep 28, 2020 at 7:18 PM Midhun Mohan wrote:
> Thank you, will check it out
>
> On Mon, Sep 28, 2020 at 7:03 PM Bryan Bende wrote:
>
>> Hello,
>>
>> Mark Payne recently published a video which relates to this t
>
> On Fri, Sep 25, 2020 at 2:57 PM Midhun Mohan
> wrote:
> >
> > Hi all,
> >
> > Lets say, If I want to configure my Nifi processor to have 100/ 200/ 500
> > concurrent tasks, what should my
> > Maximum Timer Driven Thread Count be?
> >
>
), to
make sure it is not being destroyed by my whole new setup.
Or else
If I can get a rule saying what all are depending on the having a larger
count of concurrent tasks, then it would be helpful
--
Regards,
Midhun Mohan
Hi all,
I need to know if there is a possible way to have CI/CD like pipeline setup
on NIFI.
So that I can run deployments of nifi processors to different setups of
NIFI, without even bothering about setting it up manually or trying to use
NIFI-registry.
--
Regards,
Midhun Mohan
an consume from the "failure" because it will release the queue.
>
> Good job!
>
> On Mon, Jul 20, 2020 at 12:57 PM Midhun Mohan
> wrote:
>
> > Hi Eduardo,
> > Thanks for the suggestions, I finally ended up testing it in a
> > different way.
> >
> &g
> > > If you need to confirm every HTTP call (including retries) before
> process
> > > the next in the queue, you could use a Wait/Notify. But be advised that
> > > this approach will slow down the throughput of your flow.
> > >
> > > Eduardo Fontes
>
me records.
--
Regards,
Midhun Mohan
I saw a doc
https://community.cloudera.com/t5/Support-Questions/Monitoring-nifi-back-pressure-threshold-and-trigger-an-email/td-p/218516
It shows 404 url..So I am assuming it is removed on the latest release of
NIFI
On Fri, 3 Jul 2020 at 09:26, Midhun Mohan wrote:
> Hi all,
>
> I w
to be a robust way.
Also I am using a lot of http-processors, So i would like to know do we
have a way to monitor the URL when it fails ?
--
Regards,
Midhun Mohan
he throughput of your flow.
>
> Eduardo Fontes
> Data Engineer
>
>
> On Tue, Jun 30, 2020 at 8:58 AM Midhun Mohan
> wrote:
>
> > Hi all,
> >
> > I was trying to figure out a way to do a retry on http processor on
> > following scenario
> &g
is first in first out priority.
But on my realtime load test it seems the Order is not preserved. I am
getting a mix of responses.
Please help me to figure it out
Thanks in advance
--
Regards,
Midhun Mohan
cript to hit the endpoint
> repeatedly. Something based on this:
>
> import json
> import requests
>
> requests.post(url, json = json_data_var)
>
> And then wrap it like this:
>
> $> time python benchmark.py
>
> On Wed, Mar 18, 2020 at 9:29 AM Midhun Mohan
> w
lready provided that.
>
>
>
>
> On March 18, 2020 at 09:29:27, Midhun Mohan (midhun.mo...@esginc.us)
> wrote:
>
> Hey Mike , I meant like when I try posting that many records it is taking
> bit time. Just checking did you tweak around the thread count and what is
Wed, Mar 18, 2020 at 1:29 AM Midhun Mohan
> wrote:
>
> > How did you post 50k flowfiles, that is what am looking for
> >
> > On Wed, 18 Mar 2020 at 02:36, Mike Thomsen
> wrote:
> >
> > > That's probably a lot of the issue, especially if it's an evented
&
s with 4k of JSON in them to an Express hello world app
> and it was able to respond to 50k flowfiles in under 10s using InvokeHttp
> with only 3 threads. All on my MacBook Pro.
>
> On Tue, Mar 17, 2020 at 4:54 PM Midhun Mohan
> wrote:
>
> > Endpoint does execu
> On Tue, Mar 17, 2020 at 3:55 PM Midhun Mohan
> wrote:
>
> > Yeah the endpoint which am sending right now has plenty of resources
> > available. Only thing is I need to send more records
> >
> > On Wed, 18 Mar 2020, 1:23 am Chad Zobrisky, wrote:
> >
>
ally you can increase the nifi system thread count a decent amount,
> 100+, depending on the system.
>
> On Tue, Mar 17, 2020 at 4:19 PM Midhun Mohan
> wrote:
>
> > Yeah got it thank you.
> > How will Changing that values affect the processing speed
> >
> >
r 17, 2020 at 4:11 PM Midhun Mohan
> wrote:
>
> > No overall thread pool I haven't changed. How can I change that?
> >
> > Nifi is running on a standalone aws instance. T2.medium I remember so
> that
> > leaves us with 2 cores I guess
> >
> > On Wed, 18
vel? If yes,
> to which number? How many cores do you have on NiFi?
>
> Thanks
>
> Le mar. 17 mars 2020 à 20:55, Midhun Mohan a
> écrit :
>
> > Yeah the endpoint which am sending right now has plenty of resources
> > available. Only thing is I need to send more records
vel? If yes,
> to which number? How many cores do you have on NiFi?
>
> Thanks
>
> Le mar. 17 mars 2020 à 20:55, Midhun Mohan a
> écrit :
>
> > Yeah the endpoint which am sending right now has plenty of resources
> > available. Only thing is I need to send more recor
you verified the endpoint you are sending to is not the bottle neck?
>
> Chad
>
>
> On Tue, Mar 17, 2020 at 3:47 PM Midhun Mohan
> wrote:
>
> > Size of file around kb not more than that,
> > How can I find the bottle neck,
> >
> > Yes I adjust th
thread count from the hamburger menu?
> How many threads are running total for nifi? It's the number in the top
> left of the screen.
>
> Thanks,
> Chad
>
>
>
> On Tue, Mar 17, 2020 at 3:25 PM Midhun Mohan
> wrote:
>
> > Do anyone have better idea on this
> >
> &
Do anyone have better idea on this
On Tue, 17 Mar 2020, 6:32 pm Midhun Mohan, wrote:
> Hi all,
>
> I have a queue which will be filling up with realtime records of around
> 20k records. Which is posting to an endpoint using Invokehttp processor.
>
> currently the average thr
to be processed as they receive it
thanks in advance
--
Regards,
Midhun Mohan
this link: https://pasteboard.co/IXejthc.png
>
> On Mon, Mar 2, 2020 at 9:35 AM Midhun Mohan
> wrote:
>
> > Hi Eduardo,
> > Can you please send the Image again, I am not able to open it
> >
> > On Mon, 2 Mar 2020 at 18:02, Eduardo Fontes
> > wrote:
> &g
throughput
> of your flow because the aditional steps with the Cache.
>
>
>
>
> On Sun, Mar 1, 2020 at 12:57 PM Midhun Mohan
> wrote:
>
>> So I would like to know your suggestions on how to implement wait and
>> notify.
>>
>> In my case messages
ehttp adds that ptovide high details on errors or
> success (atttibutes created/added by invokehttp to ff are prefixed with
> invoke.x
>
> Cheers,
> Emanuel O.
>
>
>
>
>
>
> On Sun, 1 Mar 2020, 12:42 Midhun Mohan, wrote:
>
> > Hi my use case is
&
eues.
>
> Please let me know what is your case.
>
> On Fri, Feb 28, 2020 at 2:39 PM Midhun Mohan
> wrote:
>
> > I think wait and notify can solve my problem. I checked the documents ,
> > which is bit unclear to me. Can anyone help me?
> >
> > Just to know few t
, 28 Feb 2020, 10:59 pm Eduardo Fontes,
wrote:
> Hi! Maybe you should try another approach, like Wait/Notify or configure
> prioritizer just before the funnel, depending on your demand.
>
> On Fri, Feb 28, 2020 at 11:40 AM Midhun Mohan
> wrote:
>
> > Hi all,
> > How
Hi all,
How to prioritize one queue before a funnel?
I need to make sure one queue is emptied before another queue is taken
over for executing, I tried with a priority attribute but it fails when a
queue with less priority has messages before the queue with high priority.
--
Regards,
Midhun
know
--
Regards,
Midhun Mohan
Hi
I have an implementation that calls the INVOKEHTTP processor more than 10
times in a minute. So if the endpoint in processor fails suddenly. Would I
be able to reprocess the failed message on a priority basis before the next
message? If so please let me know how can I do that
--
38 matches
Mail list logo
