Interestingly, whenever Jenkins agent is executing PMD/Findbugs task, it runs into " Resource temporarily unavailable". I have
- verified TCP parameters i.e. tcp_keepalive_time, tcp_keepalive_intvl, tcp_keepalive_probes & tcp_fin_timeout ; All of them have higher values then what cloudbees recommended at https://support.cloudbees.com/hc/en-us/articles/115001369667-dedicated-SSH-agents-formerly-slaves-get-disconnected - set heap space to 2-4Gb - disabled ping thread from Jenkins agent by setting "-Dhudson.remoting.Launcher.pingIntervalSec=-1" Please let me know if there is anything else I should consider. On Friday, December 11, 2020 at 12:11:39 PM UTC-5 kuisat...@gmail.com wrote: > The most common issue is related to disconnections because there is no > traffic between the Jenkins instance and the agent, for that, you have to > tune the TCP stack of your OS (see > https://support.cloudbees.com/hc/en-us/articles/115001369667-dedicated-SSH-agents-formerly-slaves-get-disconnected), > > or enable the keepalive option in the SSH protocol this can be configured > by setting ClientAliveInterval or TCPKeepAlive on the SSH server > (/etc/ssh/sshd_config), also by setting ServerAliveInterval or TCPKeepAlive > options for the user connection (/etc/ssh/ssh_config or ~/.ssh/ssh_config) > > https://www.freebsd.org/cgi/man.cgi?sshd_config(5) > https://www.freebsd.org/cgi/man.cgi?ssh_config(5) > > Also, check you follow the best practices to configure your SSH agents and > enable SSH verbose log output in your service (see > https://github.com/jenkinsci/ssh-slaves-plugin/blob/master/doc/TROUBLESHOOTING.md > ) > > El miércoles, 9 de diciembre de 2020 a las 17:53:19 UTC+1, > jiga...@gmail.com escribió: > >> Hello Jenkins community, >> >> I have setup Jenkins agents over SSH and this agent frequently goes >> offline with Resource temporarily unavailable error. I had to configure my >> Jenkins agents to communicate over JNLP. Any advise on how to fix this >> issue? >> >> Jenkins v2.249.1 >> >> [12/04/20 15:23:00] [SSH] Checking java version of java >> [12/04/20 15:23:01] [SSH] java -version returned 1.8.0_202. >> [12/04/20 15:23:01] [SSH] Starting sftp client. >> [12/04/20 15:23:03] [SSH] Remote file system root $JENKINS_SSH_DATA does >> not exist. Will try to create it... >> [12/04/20 15:23:03] [SSH] Copying latest remoting.jar... >> [12/04/20 15:23:03] [SSH] Copied 1,521,553 bytes. >> Expanded the channel window size to 4MB >> [12/04/20 15:23:03] [SSH] Starting agent process: cd "$JENKINS_SSH_DATA" >> && java -jar remoting.jar -workDir $JENKINS_SSH_DATA -jar-cache >> $JENKINS_SSH_DATA/remoting/jarCache >> Dec 04, 2020 3:23:29 PM org.jenkinsci.remoting.engine.WorkDirManager >> initializeWorkDir >> INFO: Using $JENKINS_SSH_DATA/remoting as a remoting work directory >> Dec 04, 2020 3:23:29 PM org.jenkinsci.remoting.engine.WorkDirManager >> setupLogging >> INFO: Both error and output logs will be printed to >> $JENKINS_SSH_DATA/remoting >> <===[JENKINS REMOTING CAPACITY]===>channel started >> Remoting version: 4.5 >> This is a Unix agent >> Evacuated stdout >> Agent successfully connected and online >> The Agent is connected, disconnect it before to try to connect it again. >> Dec 04, 2020 3:33:58 PM org.eclipse.jgit.util.FS discoverGitSystemConfig >> WARNING: Exception caught during execution of command '[git, config, >> --system, --edit]' in '$GIT_PATH/bin', return code '128', error message >> 'fatal: Invalid path '$GIT_PATH/etc': No such file or directory >> ' >> Dec 04, 2020 3:33:58 PM org.eclipse.jgit.util.FS$FileStoreAttributes >> saveToConfig >> WARNING: locking FileBasedConfig[$JENKINS_PATH/.config/jgit/config] >> failed after 5 retries >> Dec 04, 2020 3:33:59 PM >> org.jenkinsci.remoting.util.AnonymousClassWarnings warn >> WARNING: Attempt to (de-)serialize anonymous class >> com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTriggerBuildChooser$1; >> >> see: https://jenkins.io/redirect/serialization-of-anonymous-classes/ >> Dec 04, 2020 3:34:24 PM hudson.remoting.Request$2 run >> WARNING: Failed to send back a reply to the request >> hudson.remoting.Request$2@493c5a4e >> java.io.IOException: Resource temporarily unavailable >> at java.io.FileOutputStream.writeBytes(Native Method) >> at java.io.FileOutputStream.write(FileOutputStream.java:313) >> at >> hudson.remoting.StandardOutputStream.write(StandardOutputStream.java:83) >> at >> hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:89) >> at >> hudson.remoting.ChunkedOutputStream.sendBreak(ChunkedOutputStream.java:62) >> at >> hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:46) >> at >> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:46) >> at hudson.remoting.Channel.send(Channel.java:766) >> at hudson.remoting.Request$2.run(Request.java:388) >> at >> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> >> Dec 04, 2020 5:25:30 PM hudson.remoting.Request$2 run >> WARNING: Failed to send back a reply to the request >> hudson.remoting.Request$2@2511e2d4 >> java.io.IOException: Resource temporarily unavailable >> at java.io.FileOutputStream.writeBytes(Native Method) >> at java.io.FileOutputStream.write(FileOutputStream.java:326) >> at >> hudson.remoting.StandardOutputStream.write(StandardOutputStream.java:88) >> at >> hudson.remoting.ChunkedOutputStream.sendFrame(ChunkedOutputStream.java:90) >> at >> hudson.remoting.ChunkedOutputStream.drain(ChunkedOutputStream.java:85) >> at >> hudson.remoting.ChunkedOutputStream.write(ChunkedOutputStream.java:54) >> at java.io.OutputStream.write(OutputStream.java:75) >> at >> hudson.remoting.ChunkedCommandTransport.writeBlock(ChunkedCommandTransport.java:45) >> at >> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.write(AbstractSynchronousByteArrayCommandTransport.java:46) >> at hudson.remoting.Channel.send(Channel.java:766) >> at hudson.remoting.Request$2.run(Request.java:388) >> at >> hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >> at java.lang.Thread.run(Thread.java:748) >> >> ERROR: Connection terminated >> java.io.StreamCorruptedException: invalid stream header: 00025B42 >> at >> java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:866) >> at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358) >> at >> hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49) >> at hudson.remoting.Command.readFrom(Command.java:142) >> at hudson.remoting.Command.readFrom(Command.java:128) >> at >> hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) >> at >> hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63) >> Dec 04, 2020 5:32:30 PM hudson.slaves.ChannelPinger$1 onDead >> INFO: Ping failed. Terminating the channel channel. >> java.util.concurrent.TimeoutException: Ping started at 1607120910589 >> hasn't completed by 1607121150590 >> at hudson.remoting.PingThread.ping(PingThread.java:134) >> at hudson.remoting.PingThread.run(PingThread.java:90) >> >> Thanks, >> Jigar R >> >> -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/030aced0-ba6e-4012-a60a-58208d350544n%40googlegroups.com.
