Unfortunately, this thread landed in the vpp-dev@lists.fd.io moderation queue. I check the queue fairly often, but relying on a manual process to approve helpdesk postings is not the way to go.
It would work much better if folks managed to subscribe fdio-helpd...@rt.linuxfoundation.org to the vpp-dev and csit-dev mailing lists. Set delivery disabled, to avoid accidental ticket creation. HTH... Dave -----Original Message----- From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Vratko Polak -X via RT Sent: Tuesday, April 30, 2019 10:04 AM To: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgel...@cisco.com> Cc: csit-...@lists.fd.io; vpp-dev@lists.fd.io Subject: Re: [vpp-dev] [csit-dev] [FD.io Helpdesk #73486] Jenkins.fd.io network issues >> interleaved by quick periods of activity >>> 09:26:36 ++ sleep 197s > send any keepalive packages I always assumed the console outputs are enough to keep jnlp connection alive. Also, I believe this failure over weekend has hit multiple jobs at once. For example https://jenkins.fd.io/job/csit-vpp-perf-verify-master-3n-hsw/333/console 09:32:54 ++ sleep 184s 09:33:09 FATAL: command execution failed Vratko. -----Original Message----- From: csit-...@lists.fd.io <csit-...@lists.fd.io> On Behalf Of Kenny Paul via RT Sent: Tuesday, 2019-April-30 15:57 To: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgel...@cisco.com> Cc: csit-...@lists.fd.io; vpp-dev@lists.fd.io Subject: [csit-dev] [FD.io Helpdesk #73486] Jenkins.fd.io network issues Hello Vratko, Thank you for explanation. I'm wondering within that period of time when reservation was unsuccessful (~40min) does the job keep jnlp connection alive (send any keepalive packages)? I checked the haproxy node where jnlp is runnining and I don't see any DOWN notification for it Thanks, -- Anton Baranov Sr. System Operations Engineer The Linux Foundation On Tue Apr 30 09:27:56 2019, vrpo...@cisco.com wrote: > > 05:26:36 mkdir: cannot create directory '/tmp/reservation_dir': File > > exists > > That error is expected, it just means > the testbed is currently used by another job, so this job should sleep > a while and try again. > > > the job was waiting (sleep) from 04:45:12 til 05:26:36 > > I believe my browser is showing me UTC timestamps, which show values > larger by 4 hours. > > > we have 10m idle timeout > > The ~3m period of sleeps are interleaved by quick periods of activity, > so we usually do not hit the timeout. > > But the final sleep probably took longer for some reason > > 09:26:36 ++ sleep 197s > 09:32:20 FATAL: command execution failed > > and something bad has happened in less than 6 minutes. > So it does not look like the 10m timeout. > > Vratko. > > -----Original Message----- > From: csit-...@lists.fd.io <csit-...@lists.fd.io> On Behalf Of Kenny > Paul via RT > Sent: Tuesday, 2019-April-30 15:09 > To: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) > <jgel...@cisco.com> > Cc: csit-...@lists.fd.io; vpp-dev@lists.fd.io > Subject: [csit-dev] [FD.io Helpdesk #73486] Jenkins.fd.io network > issues > > Hello Jan > > From logs I see that the job was waiting (sleep) from 04:45:12 til > 05:26:36 which could cause jnlp session to timed out as we have 10m > idle timeout (client and server side) set on jenkins.fd.io > > Could you check that error: > > 05:26:36 Reservation unsuccessful: > 05:26:36 mkdir: cannot create directory '/tmp/reservation_dir': File > exists > > Cheers, > > -- > Anton Baranov > Sr. System Operations Engineer > The Linux Foundation > > On Mon Apr 29 02:58:28 2019, jgel...@cisco.com wrote: > > Hello, > > > > We are experiencing quite a lot of network issues when running CSIT > > tests for 19.04 report: > > > > Caused: hudson.remoting.ChannelClosedException: Channel "unknown": > > Remote call on JNLP4-connect connection from vex-yul-rot-ingress- > > 1.ci.codeaurora.org/10.30.48.3:41068 failed. The channel is closing > > down or has closed down > > > > https://jenkins.fd.io/job/csit-vpp-perf-verify-1904-3n-hsw/13/consol > > e > > > > Could you, please, have a look on it? > > > > Thank you very much. > > > > Regards, > > Jan > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12909): https://lists.fd.io/g/vpp-dev/message/12909 Mute This Topic: https://lists.fd.io/mt/31419993/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
