You should be able to set the message delimiter in ListenTCP with the "Message Delimiter" property. The default is \n. I don't believe there's a great way to see how data is being interpreted other than using interactive debug mode with NiFi set through the bootstrap.conf.
On Wed, Feb 9, 2022 at 9:29 PM Breno Cesar <[email protected]> wrote: > Hi guys, > > First of all i apologize for my english, and thank you very much for your > answers. > The data type i need is a data stream, which starts when the ipdr protocol > connect process ends. > Is there any way to change the line terminator? or somehow see how nifi > is interpreting the data when It its received ? > > Em qua., 9 de fev. de 2022 22:31, Joe Witt <[email protected]> escreveu: > >> Correct. With both ListenTCP and ListenTCPRecord it listens for data and >> needs some way of framing them to generate records/bundles of data as >> flowfiles. >> >> What kind of data do you want to listen for? >> >> Thanks >> >> On Wed, Feb 9, 2022 at 6:26 PM Michael Loftis <[email protected]> wrote: >> >>> ListenTCP isn’t fit for ingesting arbitrary data. It’s record oriented, >>> default separator is Unix newline “\n”. I don’t think it will generate a >>> flow if it never receives a newline before the connection closes. >>> >>> On Wed, Feb 9, 2022 at 18:17 Breno Cesar <[email protected]> >>> wrote: >>> >>>> i'm tring to receive a tcp stream using the listentcp module, but when >>>> the connection start, i see the traffic reaching my server (usgin tcpdump), >>>> but the nifi does note generate a flow file. >>>> Ther is no error ow warning in the module or in the nifi cluster. >>>> Changing in the processor the bulleting level to "debug" , i can see >>>> the following message for every package that i receive: >>>> >>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>>> Accepted incoming connection from /192.0.2.12:55209 >>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>>> bytes read 46 >>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] No >>>> more data available, returning for selection >>>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>>> Reached EOF, closing connection >>>> >>>> bellow is the same message, but in the tcpdump format. >>>> >>>> Internet Protocol Version 4, Src: 192.0.2.12, Dst: 192.0.2.128 >>>> Transmission Control Protocol, Src Port: 55209 , Dst Port: 4737, Seq: >>>> 1, Ack: 1, Len: 46 >>>> Source Port: 55209 >>>> Destination Port: 4737 >>>> <Source or Destination Port: 55209 > >>>> <Source or Destination Port: 4737> >>>> [Stream index: 0] >>>> [TCP Segment Len: 46] >>>> Sequence number: 1 (relative sequence number) >>>> Sequence number (raw): 3765145522 >>>> [Next sequence number: 47 (relative sequence number)] >>>> Acknowledgment number: 1 (relative ack number) >>>> Acknowledgment number (raw): 274707205 >>>> 0101 .... = Header Length: 20 bytes (5) >>>> Flags: 0x018 (PSH, ACK) >>>> Window size value: 10000 >>>> [Calculated window size: 10000] >>>> [Window size scaling factor: 1] >>>> Checksum: 0x5fd7 [unverified] >>>> [Checksum Status: Unverified] >>>> Urgent pointer: 0 >>>> [SEQ/ACK analysis] >>>> [Timestamps] >>>> TCP payload (46 bytes) >>>> [PDU Size: 46] >>>> IPDR >>>> Version: 2 >>>> Message id: CONNECT (5) >>>> Session id: 0 >>>> Message flags: 0x00 >>>> Message length: 46 >>>> Initiator id: 0.0.0.0 >>>> Initiator port: 0 >>>> Capabilities: 0x00000006 >>>> Keep-alive interval: 90 >>>> Vendor id: ARRIS E6000 Exporter >>>> >>>> Here somethings that i tried: >>>> >>>> - Increase the number of simultaneos connections and batch size, but >>>> it does not change at all. >>>> - Use listentcprecord but the behavior is the same, it only change >>>> when i select the avroreader, then i got an error informing that is a >>>> "wrong data format". >>>> >>>> So i belive that nifi is receving the messages that come from tcp , but >>>> for some reason is not generating flowfiles. >>>> >>>> Does anyone got same behavior ? >>>> There is anything else that i can do ? >>>> >>>> i'm using nifi 1.15.3 with a 4 nodes cluster. >>>> >>>> >>>> Breno >>>> >>>> >>>> >>>> -- >>> >>> "Genius might be described as a supreme capacity for getting its >>> possessors >>> into trouble of all kinds." >>> -- Samuel Butler >>> >>
