Elasticsearch dynamic index name from data attribute possible?
Hi,
We have dynamic indexes in elastic search like signal-.MM.DD => (which
resolved as signal-2017.03.31)
I was try hard to find the way to specify the dynamic index in nifi
processor.
I already have a field in my data which i want to use as dynamic index. for
example "index_name". I just wanted to use that data field in the index
parameter of nifi processor.
It might be simple, but i tried hard with different options like
'${index_name}' , "${index_name}" , '${INDEX_NAME}' , $INDEX_NAME ,
$index_name
Can anyone just refer me how to use data field in the nifi's elasticsearch
processor property of index name?
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Elasticsearch-dynamic-index-name-from-data-attribute-possible-tp15327.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Elasticsearch dynamic index name from data attribute possible?
Thanks alot Chad,
I will try to resolve through this...
On Apr 2, 2017 5:39 AM, "Chad Zobrisky [via Apache NiFi Developer List]" <
ml-node+s39713n15338...@n7.nabble.com> wrote:
> Thought about it a little more and since the flowfile is json, you could
> use the EvaluateJsonPath[1] processor to set the attribute.
>
> [1]
> https://nifi.apache.org/docs/nifi-docs/components/org.
> apache.nifi.processors.standard.EvaluateJsonPath/
>
> On Sat, Apr 1, 2017 at 4:32 PM Chad Zobrisky <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=15338&i=0>> wrote:
>
> > Hello,
> >
> > You need the index_name field to be a flow file attribute to be able to
> > use the expression language to set the index name dynamically.
> >
> > I do not think there is a processor that currently takes flow file
> content
> > and adds it as an attribute, but someone correct me if I am wrong. One
> way
> > to do so would be to use the ExecuteScript[1] processor to find the
> > index_name field in the data and set an attribute "index_name" on the
> flow
> > file. Example on how to read in flow file content and set an attribute
> in
> > python for the ExceuteScript Processor [2] .
> >
> > Then in the ElasticSearch processor you would have access to the
> > index_name attribute through the expression language, "${index_name}".
> >
> > Hope this helps,
> > Chad
> >
> > [1]
> > https://nifi.apache.org/docs/nifi-docs/components/org.
> apache.nifi.processors.script.ExecuteScript/
> > [2] http://www.nifi.rocks/using-the-executescript-processor/
> >
> > On Fri, Mar 31, 2017 at 2:33 PM shahbazatta <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=15338&i=1>>
> > wrote:
> >
> > Hi,
> > We have dynamic indexes in elastic search like signal-.MM.DD =>
> (which
> > resolved as signal-2017.03.31)
> > I was try hard to find the way to specify the dynamic index in nifi
> > processor.
> > I already have a field in my data which i want to use as dynamic index.
> for
> > example "index_name". I just wanted to use that data field in the index
> > parameter of nifi processor.
> > It might be simple, but i tried hard with different options like
> > '${index_name}' , "${index_name}" , '${INDEX_NAME}' , $INDEX_NAME ,
> > $index_name
> >
> > Can anyone just refer me how to use data field in the nifi's
> elasticsearch
> > processor property of index name?
> >
> >
> >
> > --
> > View this message in context:
> > http://apache-nifi-developer-list.39713.n7.nabble.com/
> Elasticsearch-dynamic-index-name-from-data-attribute-possible-tp15327.html
> > Sent from the Apache NiFi Developer List mailing list archive at
> > Nabble.com.
> >
> >
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-nifi-developer-list.39713.n7.nabble.com/
> Elasticsearch-dynamic-index-name-from-data-attribute-
> possible-tp15327p15338.html
> To unsubscribe from Elasticsearch dynamic index name from data attribute
> possible?, click here
> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=15327&code=c2hhaGJhei5hdHRhQGdtYWlsLmNvbXwxNTMyN3wxODE1MTg4OTg4>
> .
> NAML
> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Elasticsearch-dynamic-index-name-from-data-attribute-possible-tp15327p15340.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Elasticsearch dynamic index name from data attribute possible?
Thanks a lot Chad,
It works like charm...
On Sun, Apr 2, 2017 at 11:18 AM, Shahbaz Atta
wrote:
> Thanks alot Chad,
> I will try to resolve through this...
> On Apr 2, 2017 5:39 AM, "Chad Zobrisky [via Apache NiFi Developer List]" <
> ml-node+s39713n15338...@n7.nabble.com> wrote:
>
>> Thought about it a little more and since the flowfile is json, you could
>> use the EvaluateJsonPath[1] processor to set the attribute.
>>
>> [1]
>> https://nifi.apache.org/docs/nifi-docs/components/org.apache
>> .nifi.processors.standard.EvaluateJsonPath/
>>
>> On Sat, Apr 1, 2017 at 4:32 PM Chad Zobrisky <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=15338&i=0>> wrote:
>>
>> > Hello,
>> >
>> > You need the index_name field to be a flow file attribute to be able to
>> > use the expression language to set the index name dynamically.
>> >
>> > I do not think there is a processor that currently takes flow file
>> content
>> > and adds it as an attribute, but someone correct me if I am wrong. One
>> way
>> > to do so would be to use the ExecuteScript[1] processor to find the
>> > index_name field in the data and set an attribute "index_name" on the
>> flow
>> > file. Example on how to read in flow file content and set an attribute
>> in
>> > python for the ExceuteScript Processor [2] .
>> >
>> > Then in the ElasticSearch processor you would have access to the
>> > index_name attribute through the expression language, "${index_name}".
>> >
>> > Hope this helps,
>> > Chad
>> >
>> > [1]
>> > https://nifi.apache.org/docs/nifi-docs/components/org.apache
>> .nifi.processors.script.ExecuteScript/
>> > [2] http://www.nifi.rocks/using-the-executescript-processor/
>> >
>> > On Fri, Mar 31, 2017 at 2:33 PM shahbazatta <[hidden email]
>> <http:///user/SendEmail.jtp?type=node&node=15338&i=1>>
>> > wrote:
>> >
>> > Hi,
>> > We have dynamic indexes in elastic search like signal-.MM.DD =>
>> (which
>> > resolved as signal-2017.03.31)
>> > I was try hard to find the way to specify the dynamic index in nifi
>> > processor.
>> > I already have a field in my data which i want to use as dynamic index.
>> for
>> > example "index_name". I just wanted to use that data field in the index
>> > parameter of nifi processor.
>> > It might be simple, but i tried hard with different options like
>> > '${index_name}' , "${index_name}" , '${INDEX_NAME}' , $INDEX_NAME ,
>> > $index_name
>> >
>> > Can anyone just refer me how to use data field in the nifi's
>> elasticsearch
>> > processor property of index name?
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> > http://apache-nifi-developer-list.39713.n7.nabble.com/Elasti
>> csearch-dynamic-index-name-from-data-attribute-possible-tp15327.html
>> > Sent from the Apache NiFi Developer List mailing list archive at
>> > Nabble.com.
>> >
>> >
>>
>>
>> --
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-nifi-developer-list.39713.n7.nabble.com/Elasti
>> csearch-dynamic-index-name-from-data-attribute-possible-
>> tp15327p15338.html
>> To unsubscribe from Elasticsearch dynamic index name from data attribute
>> possible?, click here
>> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=15327&code=c2hhaGJhei5hdHRhQGdtYWlsLmNvbXwxNTMyN3wxODE1MTg4OTg4>
>> .
>> NAML
>> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Elasticsearch-dynamic-index-name-from-data-attribute-possible-tp15327p15343.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
NiFi unsecure cluster setup issue on Windows.
Hi, I am trying to setup the 3 node NiFi but i am unable to set it up. I already setup it on ubuntu and it is working fine but the same configurations not working on windows machines. I tried restarting all nodes, nifi service, etc but nothing works: Failed to determine which node is elected active Cluster Coordinator: ZooKeeper reports the address as winifi01: whereas, winifi01 node showing cluster of 1/1. i also verify the hosts file and zookeeper properties zookeer.properties server.1=winifi01:2888:3888 server.2=winifi02:2888:3888 server.3=winifi03:2888:3888 -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/NiFi-unsecure-cluster-setup-issue-on-Windows-tp15631.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
