[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
SCM/JIRA link daemon commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Code changed in jenkins User: olivier lamy Path: pom.xml http://jenkins-ci.org/commit/jenkins/8cbe3f5ab6cc6289481cc13784952802392129e5 Log: use last remoting 2.19 fix for JENKINS-6604 (cherry picked from commit 76e31a3a5c039e317a84b4c3331e15c284d44435) Conflicts: pom.xml This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Linards L edited a comment on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Seems like upgrade from v1.492 to v1.494 also fixed the ancient bug causing inability to install more than one Slave in Windows 2k8 (R2) X64 (Datacenter) using standard method .. - still - before using standard method I renamed first Slave service... This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Linards L commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Seems like this also fixed the ancient bug causing inability to install more than one Slave in Windows 2k8 (R2) X64 (Datacenter) using standard method .. - still - before using standard method I renamed first Slave service... This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Jesse Glick resolved JENKINS-6604 as Fixed Possible race condition in RemoteClassLoader renders slave unusable With @olamy’s integration, should now be fixed. Change By: Jesse Glick (03/Dec/12 4:52 PM) Status: InProgress Resolved Resolution: Fixed This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
SCM/JIRA link daemon commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Code changed in jenkins User: olivier lamy Path: pom.xml http://jenkins-ci.org/commit/jenkins/76e31a3a5c039e317a84b4c3331e15c284d44435 Log: use last remoting 2.19 fix for JENKINS-6604 This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
SCM/JIRA link daemon commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Code changed in jenkins User: olivier lamy Path: changelog.html http://jenkins-ci.org/commit/jenkins/b80789474cacc64be4954f0f4473759311e80580 Log: changelog entry for JENKINS-6604 This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Jesse Glick assigned JENKINS-6604 to Jesse Glick Possible race condition in RemoteClassLoader renders slave unusable Change By: Jesse Glick (21/Nov/12 10:19 PM) Assignee: KohsukeKawaguchi JesseGlick This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Jesse Glick started work on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Change By: Jesse Glick (21/Nov/12 10:19 PM) Status: Open InProgress This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
SCM/JIRA link daemon commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Code changed in jenkins User: Jesse Glick Path: src/main/java/hudson/remoting/RemoteClassLoader.java src/test/java/hudson/remoting/ClassRemotingTest.java src/test/java/hudson/remoting/DummyClassLoader.java src/test/java/hudson/remoting/DummyClassLoaderTest.java src/test/java/hudson/remoting/TestCallable.java http://jenkins-ci.org/commit/remoting/fdd0f4bb1bc92fb68cb2dd5d0f5a8f80e19c78d9 Log: JENKINS-6604 Race condition in RemoteClassLoader. Compare: https://github.com/jenkinsci/remoting/compare/9ac2a9e49238...fdd0f4bb1bc9 This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Jesse Glick commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable https://github.com/jenkinsci/remoting/pull/8 This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Erik Lovlie commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Ah yes, it is the same stack trace as above. Didn't notice. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Erik Lovlie commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable A new failure which could possibly have the same cause: Caused by: java.lang.LinkageError: loader (instance of hudson/remoting/RemoteClassLoader): attempted duplicate class definition for name: "hudson/model/Descriptor" at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791) at java.lang.ClassLoader.defineClass(ClassLoader.java:634) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:152) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.getDeclaredFields0(Native Method) at java.lang.Class.privateGetDeclaredFields(Class.java:2308) at java.lang.Class.getDeclaredField(Class.java:1897) at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1639) at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:69) at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:467) at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:455) at java.security.AccessController.doPrivileged(Native Method) at java.io.ObjectStreamClass.init(ObjectStreamClass.java:455) at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:352) at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:589) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1601) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369) at hudson.remoting.UserRequest.deserialize(UserRequest.java:182) at hudson.remoting.UserRequest.perform(UserRequest.java:98) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:326) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Erik Lovlie commented on JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Any chance this can be looked at? It would seem like a big severity/effort ratio. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Erik Lovlie assigned JENKINS-6604 to Kohsuke Kawaguchi Possible race condition in RemoteClassLoader renders slave unusable Change By: Erik Lovlie (02/Nov/12 1:17 PM) Assignee: AndreyMyatlyuk KohsukeKawaguchi This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Erik Lovlie updated JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Can someone explain why this bug is of "minor" priority? Comparing the consequence of this bug (dynamic slave start/stop is broken) with the cost of fixing it (applying the attached patch) I am taking the liberty of increasing the priority of this bug from "minor" to "critical". Change By: Erik Lovlie (06/Sep/12 7:39 AM) Priority: Minor Critical This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Erik Lovlie updated JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Actually, having looked at the current set of critical and blocker bugs on the "core" component, this seems like a blocker to me. It is at least blocking the use of jenkins in an environment where the slave cannot run all of the time (such as when the slave is on a cluster login box). Change By: Erik Lovlie (06/Sep/12 7:50 AM) Priority: Critical Blocker This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Erik Lovlie assigned JENKINS-6604 to Andrey Myatlyuk Possible race condition in RemoteClassLoader renders slave unusable Andrey, I'm assigning to you somewhat at random (I saw that you were working on other "core" issues). Could you look at the attached patch and consider whether it is an acceptable fix? I'm currently not able to use jenkins at my company due to this bug (dynamic slave start/stop seems broken). Change By: Erik Lovlie (06/Sep/12 7:52 AM) Assignee: AndreyMyatlyuk This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
Stevan Radakovic updated JENKINS-6604 Possible race condition in RemoteClassLoader renders slave unusable Here's the patch for the jenkins-remoting 2.12 library which should deal with this bug. Change By: Stevan Radakovic (12/Jun/12 10:23 AM) Attachment: jenkins-6604-remoting-2.12.patch This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
[ https://issues.jenkins-ci.org/browse/JENKINS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=163729#comment-163729 ] Stevan Radakovic commented on JENKINS-6604: --- Our proposed fix is in the Jenkins' remoting library, the code is available here https://code.launchpad.net/~linaro-infrastructure/jenkins-remoting/linaro-trunk . It's branched from 2.12 jenkins-remoting lib so that's something to take into consideration. We weren't able to reproduce this issue on cloned test instances but after deploying this fix, our production didn't have any more of these issues (for around 4 days now). We'll post again in a while if it doesn't happen again, and I encourage the same if someone try this fix of ours. Possible race condition in RemoteClassLoader renders slave unusable --- Key: JENKINS-6604 URL: https://issues.jenkins-ci.org/browse/JENKINS-6604 Project: Jenkins Issue Type: Bug Components: core Affects Versions: current Environment: CentOS 5.3, Sun JDK 1.6.0_19 64-bit Reporter: michal_grzejszczak Priority: Minor We are restarting hudson each Sunday afternoon to evade problems with memory leaks and have a couple of nightly builds that kick in at midnight. The scenario is that Hudson is fresh when multiple builds kick in, that is its remote class loader did not have a chance to read any classes yet. We have 3 executors defined. I suppose that the SCM poll action that is sent in many build procedures causes multiple requests to load classes for the SCM (we use slightly hacked version of CVS SCM). We are getting the following exception: java.lang.LinkageError: loader (instance of hudson/remoting/RemoteClassLoader): attempted duplicate class definition for name: hudson/model/ModelObject I have looked around on the web and found this (http://jira.codehaus.org/browse/JETTY-418) that lead me to believe that lack of synchronization while loading classes in remote class loader is the cause. Full stack trace: {code} Started on May 24, 2010 12:00:54 AM FATAL: remote file operation failed: /home/hudson-slave/workspace/BPE_8.1SR at hudson.remoting.Channel@1219b8c:slave-81 hudson.util.IOException2: remote file operation failed: /home/hudson-slave/workspace/BPE_8.1SR at hudson.remoting.Channel@1219b8c:slave-81 at hudson.FilePath.act(FilePath.java:743) at hudson.FilePath.act(FilePath.java:729) at com.syncron.hudson.cvs2.CVS2.isUpdatable(CVS2.java:813) at com.syncron.hudson.cvs2.CVS2.pollChanges(CVS2.java:310) at hudson.scm.SCM.poll(SCM.java:370) at hudson.model.AbstractProject.poll(AbstractProject.java:1153) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:330) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:359) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.IOException: Remote call on slave-81 failed at hudson.remoting.Channel.call(Channel.java:560) at hudson.FilePath.act(FilePath.java:736) ... 14 more Caused by: java.lang.LinkageError: loader (instance of hudson/remoting/RemoteClassLoader): attempted duplicate class definition for name: hudson/model/ModelObject at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.lang.ClassLoader.defineClass(ClassLoader.java:466) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.lang.ClassLoader.defineClass(ClassLoader.java:466) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131) at
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
[ https://issues.jenkins-ci.org/browse/JENKINS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=160396#comment-160396 ] Paul Sokolovsky commented on JENKINS-6604: -- We started to see these issues today with EC2 build slaves, haven't ever seen errors like this before for a year we run this infrastructure. The only change happened before today is that some additional plugins were installed. We're still on Jenkins 1.419. Possible race condition in RemoteClassLoader renders slave unusable --- Key: JENKINS-6604 URL: https://issues.jenkins-ci.org/browse/JENKINS-6604 Project: Jenkins Issue Type: Bug Components: core Affects Versions: current Environment: CentOS 5.3, Sun JDK 1.6.0_19 64-bit Reporter: michal_grzejszczak Priority: Minor We are restarting hudson each Sunday afternoon to evade problems with memory leaks and have a couple of nightly builds that kick in at midnight. The scenario is that Hudson is fresh when multiple builds kick in, that is its remote class loader did not have a chance to read any classes yet. We have 3 executors defined. I suppose that the SCM poll action that is sent in many build procedures causes multiple requests to load classes for the SCM (we use slightly hacked version of CVS SCM). We are getting the following exception: java.lang.LinkageError: loader (instance of hudson/remoting/RemoteClassLoader): attempted duplicate class definition for name: hudson/model/ModelObject I have looked around on the web and found this (http://jira.codehaus.org/browse/JETTY-418) that lead me to believe that lack of synchronization while loading classes in remote class loader is the cause. Full stack trace: {code} Started on May 24, 2010 12:00:54 AM FATAL: remote file operation failed: /home/hudson-slave/workspace/BPE_8.1SR at hudson.remoting.Channel@1219b8c:slave-81 hudson.util.IOException2: remote file operation failed: /home/hudson-slave/workspace/BPE_8.1SR at hudson.remoting.Channel@1219b8c:slave-81 at hudson.FilePath.act(FilePath.java:743) at hudson.FilePath.act(FilePath.java:729) at com.syncron.hudson.cvs2.CVS2.isUpdatable(CVS2.java:813) at com.syncron.hudson.cvs2.CVS2.pollChanges(CVS2.java:310) at hudson.scm.SCM.poll(SCM.java:370) at hudson.model.AbstractProject.poll(AbstractProject.java:1153) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:330) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:359) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.IOException: Remote call on slave-81 failed at hudson.remoting.Channel.call(Channel.java:560) at hudson.FilePath.act(FilePath.java:736) ... 14 more Caused by: java.lang.LinkageError: loader (instance of hudson/remoting/RemoteClassLoader): attempted duplicate class definition for name: hudson/model/ModelObject at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.lang.ClassLoader.defineClass(ClassLoader.java:466) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.lang.ClassLoader.defineClass(ClassLoader.java:466) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.Class.getDeclaredFields0(Native Method) at java.lang.Class.privateGetDeclaredFields(Class.java:2291) at java.lang.Class.getDeclaredField(Class.java:1880) at
[JIRA] (JENKINS-6604) Possible race condition in RemoteClassLoader renders slave unusable
[ https://issues.jenkins-ci.org/browse/JENKINS-6604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=160283#comment-160283 ] Simon Schlachter commented on JENKINS-6604: --- Same Issue here. Running Jenkins ver. 1.437 Possible race condition in RemoteClassLoader renders slave unusable --- Key: JENKINS-6604 URL: https://issues.jenkins-ci.org/browse/JENKINS-6604 Project: Jenkins Issue Type: Bug Components: core Affects Versions: current Environment: CentOS 5.3, Sun JDK 1.6.0_19 64-bit Reporter: michal_grzejszczak Priority: Minor We are restarting hudson each Sunday afternoon to evade problems with memory leaks and have a couple of nightly builds that kick in at midnight. The scenario is that Hudson is fresh when multiple builds kick in, that is its remote class loader did not have a chance to read any classes yet. We have 3 executors defined. I suppose that the SCM poll action that is sent in many build procedures causes multiple requests to load classes for the SCM (we use slightly hacked version of CVS SCM). We are getting the following exception: java.lang.LinkageError: loader (instance of hudson/remoting/RemoteClassLoader): attempted duplicate class definition for name: hudson/model/ModelObject I have looked around on the web and found this (http://jira.codehaus.org/browse/JETTY-418) that lead me to believe that lack of synchronization while loading classes in remote class loader is the cause. Full stack trace: {code} Started on May 24, 2010 12:00:54 AM FATAL: remote file operation failed: /home/hudson-slave/workspace/BPE_8.1SR at hudson.remoting.Channel@1219b8c:slave-81 hudson.util.IOException2: remote file operation failed: /home/hudson-slave/workspace/BPE_8.1SR at hudson.remoting.Channel@1219b8c:slave-81 at hudson.FilePath.act(FilePath.java:743) at hudson.FilePath.act(FilePath.java:729) at com.syncron.hudson.cvs2.CVS2.isUpdatable(CVS2.java:813) at com.syncron.hudson.cvs2.CVS2.pollChanges(CVS2.java:310) at hudson.scm.SCM.poll(SCM.java:370) at hudson.model.AbstractProject.poll(AbstractProject.java:1153) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:330) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:359) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.IOException: Remote call on slave-81 failed at hudson.remoting.Channel.call(Channel.java:560) at hudson.FilePath.act(FilePath.java:736) ... 14 more Caused by: java.lang.LinkageError: loader (instance of hudson/remoting/RemoteClassLoader): attempted duplicate class definition for name: hudson/model/ModelObject at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.lang.ClassLoader.defineClass(ClassLoader.java:466) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.lang.ClassLoader.defineClass(ClassLoader.java:466) at hudson.remoting.RemoteClassLoader.loadClassFile(RemoteClassLoader.java:151) at hudson.remoting.RemoteClassLoader.findClass(RemoteClassLoader.java:131) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at java.lang.Class.getDeclaredFields0(Native Method) at java.lang.Class.privateGetDeclaredFields(Class.java:2291) at java.lang.Class.getDeclaredField(Class.java:1880) at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1610) at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:52) at