I was able to construct the Json with right data type output from ExecuteScript and sending to Kafka directly. The problem i am seeing is if i send one record to kafka, kafka processor is writing the message again and again and not ending the loop. How can i send exactly once message? Any help.
Here is what i am doing in my script: import simplejson as json from org.apache.nifi.processor.io import StreamCallback from user_agents import parse def num(s): try: return int(s) except ValueError: try: return float(s) except ValueError: raise ValueError('argument is not a string of number') class PyStreamCallback(StreamCallback): def __init__(self): pass def process(self, inputStream, outputStream): obj = {'browser': str(parse(flowFile.getAttribute('useragent')).browser.family), 'browser_version': str(parse(flowFile.getAttribute('useragent')).browser.version_string), 'os': str(parse(flowFile.getAttribute('useragent')).os.family), 'os_version': str(parse(flowFile.getAttribute('useragent')).os.version_string), 'client_ip': flowFile.getAttribute('clientip')} if flowFile.getAttribute('http.param.t_resp') and flowFile.getAttribute('http.param.t_page') and flowFile.getAttribute('http.param.t_done'): obj['rt_firstbyte'] = num(flowFile.getAttribute('http.param.t_resp')) obj['rt_lastbyte'] = num(flowFile.getAttribute('http.param.t_page')) obj['rt_loadtime'] = num(flowFile.getAttribute('http.param.t_done')) outputStream.write(bytearray(json.dumps(obj, indent=4).encode('utf-8'))) flowFile = session.get() if (flowFile != None): flowFile = session.write(flowFile, PyStreamCallback()) session.transfer(flowFile, REL_SUCCESS) Thanks On Tue, Mar 29, 2016 at 2:30 AM, Conrad Crampton < conrad.cramp...@secdata.com> wrote: > Hi, > Depending on the final destination of the data (json) you could use the > JsonToAvro -> ConvertAvroSchema -> AvroToJson, with the ConvertAvroSchema > doing the type conversion. I had to do this as I came across this behaviour > previously. I use the Avro directly (after the conversion) as that was my > final data format requirement, but I don’t see any reason if you want Json > back that this wouldn’t work. I haven’t tried this by the way, but the type > conversion certainly works for the final attributes in the Avro documents. > Conrad > > From: Madhukar Thota <madhukar.th...@gmail.com> > Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> > Date: Friday, 25 March 2016 at 14:01 > To: "users@nifi.apache.org" <users@nifi.apache.org> > Subject: Re: String conversion to Int, float double > > Any Other ways to achieve this? > > On Thu, Mar 24, 2016 at 4:48 PM, Bryan Bende <bbe...@gmail.com> wrote: > >> I think the problem is that all attributes are actually Strings >> internally, even after calling toNumber() that is only temporary while the >> expression language is executing. >> >> So by the time it gets to AttributesToJson it doesn't have any >> information about the type of each attribute and they all end up as >> Strings. I think we would have to come up with a way to pass some type >> information along to AttributesToJson in order to get something other than >> Strings. >> >> -Bryan >> >> >> On Thu, Mar 24, 2016 at 3:30 PM, Madhukar Thota <madhukar.th...@gmail.com >> > wrote: >> >>> Hi i am trying to convert string value to integer in UpdateAtrributes >>> using toNumber like this >>> >>> >>> ${http.param.t_resp:toNumber()} where http.param.t_resp = "132" >>> >>> but when the fileattribute pushed to Attributetojson processor , i am >>> stilling seeing it as string. Am i am doing something wrong? and also how >>> can i convert string to float? >>> >>> >>> >>> >>> >> > > > ***This email originated outside SecureData*** > > Click here <https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==> to > report this email as spam. > > > SecureData, combating cyber threats > > ------------------------------ > > The information contained in this message or any of its attachments may be > privileged and confidential and intended for the exclusive use of the > intended recipient. If you are not the intended recipient any disclosure, > reproduction, distribution or other dissemination or use of this > communications is strictly prohibited. The views expressed in this email > are those of the individual and not necessarily of SecureData Europe Ltd. > Any prices quoted are only valid if followed up by a formal written quote. > > SecureData Europe Limited. Registered in England & Wales 04365896. > Registered Address: SecureData House, Hermitage Court, Hermitage Lane, > Maidstone, Kent, ME16 9NT >