Put data to Elastic with static settings or index template

[Bobby]

Hi, when inserting data to elastic using nifi's processor (putElastic), i
need to apply static setting for the index..like mentioned in 
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html

 
, this must be applied in index creation..

With the processor, will it be possible to use this utility? I need to do
this in order to save the space...or in other word, changing the compression
type...

As for last resort, i might need to write custom processor extended from
putElastic 


Any suggestion?

Thank you



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Need opinion, NIFI for real time stream processing??

[Bobby]

Thanks a lot, i managed to create a solid poc for my project..Although batch
processing is nice, but client's requirement need to do stream processing
instead...
I actually have another problem yet to ask but i will open it in another
thread since it has different context..

Thank you again for your suggestion Joe 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: A user's experience with a bake off

[Joe Witt]

Mike,

What specifically do you have in mind?  There are a lot of good/useful
things we can do of course.  For instance we can make records a first
class part of the nifi-api and then provide live monitoring over them.
We have tons of good data in that regard.  But, we must also be
mindful of the need to make progress on the extension registry as
well.

Thanks

On Mon, May 21, 2018 at 4:17 PM, Mike Thomsen  wrote:
> https://statsbot.co/blog/open-source-etl/
>
> Someone on my team shared that. Overall, it's a very fair take. I think it
> has some direct action items, particularly on the UX side that could be
> considered for 1.8 to close some gaps.
>
> (Not trying to start a flame war here, as once again I think it's a fair
> article)
>
> Mike

[RESULT] [VOTE] Change the name Apache NiFi Fluid Design System to Apache NiFi Flow Design System

[Rob Moran]

Thanks to all who participated. The results are:

+1: 12 (10 binding, 2 non-binding​)
+0:  none
-1:  none

The vote to change the name Apache NiFi Fluid Design System to Apache NiFi
Flow Design System has passed.

I will file a JIRA to update the nifi-fds README.

Thanks,
Rob

Re: NiFi ListenTCP 1.4.0 "Rejecting Connections from xx.xx.xx.xx : xxxx, because max connection has been met" timeout

[Bryan Bende]

The max connection warning is simply based on the property in the
processor called "Max Number of TCP Connections" which defaults to 2.
So in the default case, if a third connection is made while two
connections are open, then the third connection is rejected and you
see this warning.

If you are saying that the issue eventually corrects it self, then
maybe just increase the max allowed connections slightly.

If that is not helping, are there any other errors in the NiFi logs
around the time of connection loss? (besides the rejected connections
that happen later).

-Bryan


On Mon, May 21, 2018 at 3:36 AM, yi  wrote:
> Hi there,
>
> I am using a ListenTCP to capture data coming from an Arduino on a TCP port
> over WiFi.
>
> When there is a loss of comms over wifi for various reasons (loss of signal,
> power cut), the Arduino is programmed to attempt to connect back when
> connection is restored.
>
> I am seeing at times the max connections warning as the arduino attemps to
> reconnect, and it is unable to regain data flow for a long time - it seems
> to be a random amount of time from a few minutes up to half an hour.
>
> I have had a look at this article
>
> https://community.hortonworks.com/questions/90478/how-to-release-occupied-connections-in-listentcp-p.html
>
> which suggests that the Arduino side should close the connection, however
> during an abrupt connection loss, NiFi would not be able to detect a
> closure.
>
> Does anybody know if there is a set timeout period on the max connections
> condition?
>
> Is there a way to control how the max connection warning is triggered?
>
> Thank you
>
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/