Hi Robert, Thanks for providing the additional details. It should be possible to replace the current version of OkHttp 4.9.1 with an older version to see if that makes a difference. It would also be helpful to know whether the remote server supports HTTP/2. Newer versions of OkHttp have improved support for HTTP/2, but it also has different connection characteristics. Setting the Disable HTTP/2 property to True in InvokeHTTP would force the use of HTTP/1.1. I would not necessarily expect to see errors on the server side, but knowing whether the remote server has a connection or write timeout property would be useful.
Regards, David Handermann On Sun, May 30, 2021 at 4:54 AM Robert R. Bruno <rbru...@gmail.com> wrote: > When seeing the error we put our timeouts values in the processor both to > 5 mins as a test and still saw the errors and well before 5 minutes. We > also slowed the processor down a lot and still were seeing the error. > Failed attempts will often succeed just fine but not always. > > As an easy test could we just rebuild with the older http client library > or did a lot more change with the processor? > > We do have access to both endpoints and plan to dig deeper there as well, > but initial looking did not show errors on server side. > > On Sat, May 29, 2021, 23:26 David Handermann <exceptionfact...@apache.org> > wrote: > >> Hi Robert, >> >> It would be helpful to know the settings for the Read Timeout and Idle >> Timeout properties on the InvokeHTTP processors. If you have access to the >> remote service being called, it would also be interesting to know if there >> are timeouts on that side of the connection. NiFi 1.13.2 includes a much >> newer version of the OkHttp client library, which InvokeHTTP uses, so the >> internal connection handling has gone through some changes. In general, >> broken pipe errors suggest that the connection is timing out at some point, >> which may be related to a number of a variety of factors such as the number >> of connections, payload sizes, network latency, or local resource >> consumption. >> >> Regards, >> David Handermann >> >> On Sat, May 29, 2021 at 2:08 PM Joe Witt <joe.w...@gmail.com> wrote: >> >>> K. We have seen specific jvm versions causing issues with socket >>> handling. But had not seen it on Java 11 though may be possible. Is >>> there a full stack trace? >>> >>> On Sat, May 29, 2021 at 12:00 PM Robert R. Bruno <rbru...@gmail.com> >>> wrote: >>> >>>> We upgraded to java 11 when we upgrade to 1.13.2 we were on java 8 with >>>> 1.9.2. >>>> >>>> On Sat, May 29, 2021, 14:21 Joe Witt <joe.w...@gmail.com> wrote: >>>> >>>>> What JVM are you using? >>>>> >>>>> Thanks >>>>> >>>>> On Sat, May 29, 2021 at 11:16 AM Juan Pablo Gardella < >>>>> gardellajuanpa...@gmail.com> wrote: >>>>> >>>>>> Not related to Nifi, but I faced the same type of issue for endpoints >>>>>> behind a proxy which takes more than 30 seconds to answer. Fixed by >>>>>> replacing Apache Http client by OkHttp. I did not investigate further, >>>>>> just >>>>>> simply replaced one library by another and the error was fixed. >>>>>> >>>>>> >>>>>> Juan >>>>>> >>>>>> On Sat, 29 May 2021 at 15:08, Robert R. Bruno <rbru...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I wanted to see if anyone has any ideas on this one. Since >>>>>>> upgrading to 1.13.2 from 1.9.2 we are starting to see broken pipe (write >>>>>>> failed) errors from a few invokeHttp processers. >>>>>>> >>>>>>> It is happening to processors talking to different endpoints, so I >>>>>>> am suspecting it is on the nifi side. We are now using load balanced >>>>>>> queues throughout our flow. Is it possible we are hitting a http >>>>>>> connection resource issue or something like that? A total guess I'll >>>>>>> admit. >>>>>>> >>>>>>> If this could be it, does anyone know which parameter(s) to play >>>>>>> with in the properties file? I know there is one setting for jetty >>>>>>> threads >>>>>>> and another for max concurrent requests, but it isn't quite clear to me >>>>>>> of >>>>>>> they are at all involved with invokeHttp calls. >>>>>>> >>>>>>> Thanks in advance! >>>>>>> >>>>>>> Robert >>>>>>> >>>>>>
