NIfi relace text for XML flow file content

[Puspak Das]

Hi team ,

I have a xml flow file content where an particular tag is under there inside 
several parent tag .
but i just want to replace one particular tag value with an attribute value of 
the same flow file .

Please suggest how to achive it .

Regards
Puspak

Need Help on Nifi replace text with regex

[Puspak]

Hi Team ,

I am relatively new to Nifi regex .

i have a requirement like i have a flow file content as below .


KH228
20JUN19
Y

And the expected out put is 

  KH228
  20JUN19


i know we can achieve the same  with replace text and nifi expression
language but not sure how .

Quick help is highly appriciated. 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Evaluate json path with destination as flowfile-attribute returns null for a specified attribute

[Puspak]

Thanks Dene for the quick response .

I have achieved the same by some other means. 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Evaluate json path with destination as flowfile-attribute returns null for a specified attribute

[Puspak]

hi
The content of the flow file is as below .

{  
   "REQUEST":{  
  "SERVICENAME":"WTX_SERV_ECSESC",
  "WSTOKEN":"ZEegEneBot59",
  "ECSESC-INP":{  
 "SVCHEADER":{  
"STATION":"OGG",
"OFFICE":"001",
"AGENT":"255"
 },
 "SVCACTION":"RETRIEVE",
 "PPP-INFO":{  
"PPP":"68700200220",
"KEYINFO":{  
   "DBKEY":"113313412401",
   "REVISION":"17"
},
"CRA-RATING":"Y"
 }
  }
   }
}

and i have set the value for pppNumber as
$."REQUEST"."ECSESC-INP"."AWB-INFO"."AWB" in the evaluate json path and set
the destination as flow-file-attribute , but i am getting the value for
pppNumber  as null 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Message sending to JMS through MQ gives error-Transport scheme NOT recognized:

[Puspak]

Hi , 

I am relatively new to nifi .I have a requirment where i have to push some
messages to JMS (HornetQ).

When i push the message with below configuration in putJMS of nifi i am
getting the below error.

2019-06-13 09:30:31,701 ERROR [Timer-Driven Process Thread-7]
o.apache.nifi.processors.standard.PutJMS
PutJMS[id=4a3d7305-016b-1000-c244-a62fe67418b0] Failed to connect to JMS
Server due to javax.jms.JMSException: Could not create Transport. Reason:
java.io.IOException: Transport scheme NOT recognized: [http]:
javax.jms.JMSException: Could not create Transport. Reason:
java.io.IOException: Transport scheme NOT recognized: [http]
javax.jms.JMSException: Could not create Transport. Reason:
java.io.IOException: Transport scheme NOT recognized: [http]
at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
at
org.apache.activemq.ActiveMQConnectionFactory.createTransport(ActiveMQConnectionFactory.java:333)
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:346)
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:304)
at
org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:244)

 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Json conversion query : Help

[Puspak]

Thank you



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Json conversion query : Help

[Puspak]

Thanks Matt for such a quick response,

This perfectly works for the above scenario .
But my requirement is little different . I tried to implement the same in my
case but failed .Could you please suggest . This is because I am relatively
new to jolt.

input : 
{
  "eta" : "1557840900",
  "actualdeparturetime" : "0",
  "etd" : "1557835200",
  "ident" : "JST529",
  "FlightNumber" : "529",
  "CarrierCode" : "JQ",
  "actualarrivaltime" : "1557840900",
  "startDate" : "1557772200"
}
output :
{
  "ident" : "JST529",
  "eta" : {
"$date" : 155784090
  },
  "actualdeparturetime" : "0",
  "etd" : {
"$date" : 155783520
  },
  "messageType" : "FFM",
  "FlightNumber" : "529",
  "CarrierCode" : "JQ",
  "actualarrivaltime" : "0",
  "startDate" : "1557772200"
}

that means basically the json has some other few fields as well 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: How to convert unix timestamp to datetime in Apache NiFi

[Puspak]

Thanks for the suggestion 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

How to convert unix timestamp to datetime in Apache NiFi

[Puspak]

i have an attribute named "actualarrivaltime"  and value as
101600(unixmiliseconds)
i want to covert the same to a date fromat .

i tried ${actualarrivaltime : format('MM/dd/ HH:mm:ss.SSS')}, but it
gives me wrong result .

i am getting the value after the conversion is actualarrivaltime :01/01/1970
05:30:00.000



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: How to pass a flowfile attribute into the query of a getMongo

[Puspak]

Thanks for the response



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

How to pass a flowfile attribute into the query of a getMongo

[Puspak]

hi 

How to pass a flowfile attribute into the query of a getMongo in nifi to
fetch a particular record



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: InvokeHttp - Response - Not readable

[Puspak]

i tried with that user defined attribute as well earlier, but no luck :(



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: InvokeHttp - Response - Not readable

[Puspak]

Thanks Bryan ,

I sent the floe file to a putfile and read it from there .the content looks
like below .




This portal requires browser which support 
JavaScript.
Please make sure that you are using an updated browser and that
JavaScript is enabled.




and please find the below properties those are set for my invokehttp .If
possible please let me know i something wrong over there .i donot have a
property Accept:application/json now but still the same issue.

HTTP Method:GET
Remote
URL:http://somewebsite.something.com/json/someXML2/someInfoEx?somevalue=MO234=1=0
 
SSL Context Service:No value set
Connection Timeout:5 secs
Read Timeout:15 secs
Include Date Header:True
Follow Redirects:True
Attributes to Send:No value set
Basic Authentication Username:poc
Basic Authentication Password:Sensitive value set
Proxy Configuration Service:No value set
Proxy Host:No value set
Proxy Port:No value set
Proxy Type:http
Proxy Username:No value set
Proxy Password:No value set
Put Response Body In Attribute:No value set
Max Length To Put In Attribute:2048
Use Digest Authentication:false
Always Output Response:false
Trusted Hostname:No value set
Add Response Headers to Request:false
Content-Type:${mime.type}
Send Message Body:true
Use Chunked Encoding:false
Penalize on "No Retry":false
Use HTTP ETag:false
Maximum ETag Cache Size:10MB



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: InvokeHttp - Response - Not readable

[Puspak]

Thanks Bryan for the quick response .

Below is the complete details of the response .
Attribute Values
Cache-Control
no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection
close
Content-Length
294
Content-Type
text/html; charset=utf-8
Date
Mon, 08 Apr 2019 11:08:25 GMT
Expires
Thu, 19 Nov 1981 08:52:00 GMT
Pragma
no-cache
RouteOnAttribute.Route
FFM
RouteOnContent.Route
unmatched
Server
CPWS
Set-Cookie
NACSID=d2c5350a1510b4fed30acf87207b32ab; path=/; secure; HttpOnly
filename
232-62998983.json
fragment.count
12
fragment.identifier
b3b1da21-328d-4327-8074-a270519190cf
fragment.index
4
http.context.identifier
3ab5f934-234c-429b-865a-dced0a594aed
http.headers.Accept
*/*
http.headers.Accept-Encoding
gzip, deflate
http.headers.Cache-Control
no-cache
http.headers.Connection
keep-alive
http.headers.Content-Length
765
http.headers.Content-Type
text/plain
http.headers.Host
hostname.com:9990
http.headers.Postman-Token
4da38cf1-f2f3-40a3-866f-b59e15ccf9ec
http.headers.User-Agent
PostmanRuntime/7.6.0
http.local.addr
143.261.50.23
http.local.name
159.521.40.23
http.locale
en_US
http.method
POST
http.protocol
HTTP/1.1
http.remote.addr
159.521.40.23
http.remote.host
159.521.40.23
http.request.uri
/lll/ddd/xxx/msg
http.request.url
http://hostname.com:9990/lll/ddd/xxx/msg
http.server.name
hostname
http.server.port
9990
http.servlet.path
Empty string set
invokehttp.request.url
http://somewebsite.something.com/json/someXML2/someInfoEx?somevalue=MO234=1=0
 
invokehttp.status.code
200
invokehttp.status.message
OK
invokehttp.tx.id
0f7e2049-ae0f-43b6-a439-00ce86dfc7d9
mime.type
text/html; charset=utf-8
msgFormat
EDT
msgType
FGF
path
./
segment.original.filename
08062ff0-010d-45a6-a27e-56709a6f441e.json
uuid
e985bd29-2a05-4060-a214-3018f11c697f



The response code is 200 , and i am able to access it through postman
perfectly . Gives me desired output



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

InvokeHttp - Response - Not readable

[Puspak]

Hi team ,
My invokehttp is making a get call over http , but the response i am getting
not able to read .

when i am reading the flow file i can only see "No viewer is registered for
this content type."  I tried various option to get the json response , but
no luck.

approach i tried : 
1 Accept : application/json
2 AlwaysoutputResponse : Content-type= application/json

Please suggest  



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

GetHttp always givues error "SocketTimeotException,Read timedOut"

[Puspak]

 

GetHttp always givues error "SocketTimeotException,Read timedOut" .
This issue is despite of the remote url it is connecting is always up and
running . 

GetHttpError.png

  




--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: How to remove a specific pattern from the flow file through replace text in nifi by using regex

[Puspak]

many thanks Peter,it solution works fine 



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

How to remove a specific pattern from the flow file through replace text in nifi by using regex

[Puspak]

Hi i am relatively new to nifi and expression language .Please help

Sample flow file :

=HEADER
RCV,2019/02/27 08:34
=PRIORITY
QK
270834
=SMI
FFD
=TEXT
FFD/8
1/BY0077/27FEB1045/CMB/GB/27FEB1200/LGW
ULD/PMC74349R7
612-56018325CMBLHR/T130K1172.50MC10.08/CONSOL
/DHC
LAST

The desired flow file after replacement is :
FFD/8
1/BY0077/27FEB1045/CMB/GB/27FEB1200/LGW
ULD/PMC74349R7
612-56018325CMBLHR/T130K1172.50MC10.08/CONSOL
/DHC
LAST

basically i just want to trim-out the below portion from the flow file.

=HEADER
RCV,2019/02/27 08:34
=PRIORITY
QK
270834
=SMI
FFD
=TEXT



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Not able to connect through site-to-site remote professor group to the last nifi in a nifi chaining

[Puspak]

Thank a lot Koji for the response .

The issue was because of the trailing white space as you have mentioned .

Regards



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Not able to connect through site-to-site remote professor group to the last nifi in a nifi chaining

[Puspak]

# Site to Site properties-for Nifi1
# nifi.remote.input.host=
# nifi.remote.input.secure=false
# nifi.remote.input.socket.port=8443
# nifi.remote.input.http.enabled=true
# nifi.remote.input.http.transaction.ttl=30 sec
# nifi.remote.contents.cache.expiration=30 secs
===
# Site to Site properties- for Nifi2
nifi.remote.input.host=FQDN of Nifi2
nifi.remote.input.secure=false
nifi.remote.input.socket.port=8443
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.contents.cache.expiration=30 secs

# Site to Site properties- for Nifi3
nifi.remote.input.host=FQDN of Nifi3
nifi.remote.input.secure=false
nifi.remote.input.socket.port=
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
=
Basically i have 3 nifi in chain Nifi1--> Nifi2-->Nifi3 connected via
remoteproceddor group and i have the site-to-site properties configured as
above.
I am getting the connectivity and push message successfully from
Nifi1->Nifi2 , but from nifi2->nifi3 i am not able to get the connectivity
and push message .

I am getting the below error when trying to push message from Nifi2->Nifi3

2019-03-02 07:41:06,181 WARN [Timer-Driven Process Thread-4]
o.a.n.controller.tasks.ConnectableTask Administratively Yielding
RemoteGroupPort[name=from ind23 Nifi,targets=http://FQDN-OF-NIFI3:8080/nifi]
due to uncaught Exception: java.lang.RuntimeException:
java.lang.IllegalArgumentException: Invalid URL: http://FQDN-OF-NIFI3
:8080/nifi-api
java.lang.RuntimeException: java.lang.IllegalArgumentException: Invalid URL:
http://FQDN-OF-NIFI3 :8080/nifi-api
at 
org.apache.nifi.controller.AbstractPort.onTrigger(AbstractPort.java:257)




--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/