Hi guys, First of all i apologize for my english, and thank you very much for your answers. The data type i need is a data stream, which starts when the ipdr protocol connect process ends. Is there any way to change the line terminator? or somehow see how nifi is interpreting the data when It its received ?
Em qua., 9 de fev. de 2022 22:31, Joe Witt <[email protected]> escreveu: > Correct. With both ListenTCP and ListenTCPRecord it listens for data and > needs some way of framing them to generate records/bundles of data as > flowfiles. > > What kind of data do you want to listen for? > > Thanks > > On Wed, Feb 9, 2022 at 6:26 PM Michael Loftis <[email protected]> wrote: > >> ListenTCP isn’t fit for ingesting arbitrary data. It’s record oriented, >> default separator is Unix newline “\n”. I don’t think it will generate a >> flow if it never receives a newline before the connection closes. >> >> On Wed, Feb 9, 2022 at 18:17 Breno Cesar <[email protected]> >> wrote: >> >>> i'm tring to receive a tcp stream using the listentcp module, but when >>> the connection start, i see the traffic reaching my server (usgin tcpdump), >>> but the nifi does note generate a flow file. >>> Ther is no error ow warning in the module or in the nifi cluster. >>> Changing in the processor the bulleting level to "debug" , i can see the >>> following message for every package that i receive: >>> >>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>> Accepted incoming connection from /192.0.2.12:55209 >>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>> bytes read 46 >>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] No >>> more data available, returning for selection >>> SPOLPAVMW131:8080ListenTCP[id=e083186d-017e-1000-0000-000040efa66e] >>> Reached EOF, closing connection >>> >>> bellow is the same message, but in the tcpdump format. >>> >>> Internet Protocol Version 4, Src: 192.0.2.12, Dst: 192.0.2.128 >>> Transmission Control Protocol, Src Port: 55209 , Dst Port: 4737, Seq: >>> 1, Ack: 1, Len: 46 >>> Source Port: 55209 >>> Destination Port: 4737 >>> <Source or Destination Port: 55209 > >>> <Source or Destination Port: 4737> >>> [Stream index: 0] >>> [TCP Segment Len: 46] >>> Sequence number: 1 (relative sequence number) >>> Sequence number (raw): 3765145522 >>> [Next sequence number: 47 (relative sequence number)] >>> Acknowledgment number: 1 (relative ack number) >>> Acknowledgment number (raw): 274707205 >>> 0101 .... = Header Length: 20 bytes (5) >>> Flags: 0x018 (PSH, ACK) >>> Window size value: 10000 >>> [Calculated window size: 10000] >>> [Window size scaling factor: 1] >>> Checksum: 0x5fd7 [unverified] >>> [Checksum Status: Unverified] >>> Urgent pointer: 0 >>> [SEQ/ACK analysis] >>> [Timestamps] >>> TCP payload (46 bytes) >>> [PDU Size: 46] >>> IPDR >>> Version: 2 >>> Message id: CONNECT (5) >>> Session id: 0 >>> Message flags: 0x00 >>> Message length: 46 >>> Initiator id: 0.0.0.0 >>> Initiator port: 0 >>> Capabilities: 0x00000006 >>> Keep-alive interval: 90 >>> Vendor id: ARRIS E6000 Exporter >>> >>> Here somethings that i tried: >>> >>> - Increase the number of simultaneos connections and batch size, but it >>> does not change at all. >>> - Use listentcprecord but the behavior is the same, it only change when >>> i select the avroreader, then i got an error informing that is a "wrong >>> data format". >>> >>> So i belive that nifi is receving the messages that come from tcp , but >>> for some reason is not generating flowfiles. >>> >>> Does anyone got same behavior ? >>> There is anything else that i can do ? >>> >>> i'm using nifi 1.15.3 with a 4 nodes cluster. >>> >>> >>> Breno >>> >>> >>> >>> -- >> >> "Genius might be described as a supreme capacity for getting its >> possessors >> into trouble of all kinds." >> -- Samuel Butler >> >
