On 18 March 2018 at 03:21, Sean Talts <sean.ta...@columbia.edu> wrote: > Hey all, > > I'm having this sudden crazy problem where my Windows agent can't delete > some files in its workspace anymore, failing all builds. It's getting a > java.nio.file.AccessDeniedException on some files that it created. You can > see the full exceptions at the end of the log here: > http://d1m1s1b1.stat.columbia.edu:8080/job/Stan/job/develop/80/console also > copied below. > > I've tried so many things, including switching JVMs on master and agent, > upgrading and switching to LTS, uninstalling and reinstalling a new agent > service in a new working directory… Any tips or advice would be extremely > appreciated; been trying to solve this all day and night since it started > Thursday when the master rebooted during a job on the Windows agent.
We have thins kind of errors on Windows when there are still running processes that have a file descriptor opened. You could log on to the machine and check if there are any "zombie" processes. Chris > > Thanks in advance, > Sean > > Full error: > > java.nio.file.AccessDeniedException: > C:\Jenkins2\workspace\Stan_develop-3CVGV42HAM7J3BLI3M44PCR6M52FN6ZQ65IEZ6TOSHJURSBO2PPA\src\test\test-models\bad\read_only > at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) > at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) > at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) > at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source) > at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(Unknown Source) > at java.nio.file.Files.deleteIfExists(Unknown Source) > at hudson.Util.tryOnceDeleteFile(Util.java:297) > at hudson.Util.deleteFile(Util.java:253) > Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to > JNLP4-connect connection from > gelman-group-win.stat.columbia.edu/128.59.76.64:50229 > at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1737) > at hudson.remoting.UserResponse.retrieve(UserRequest.java:313) > at hudson.remoting.Channel.call(Channel.java:952) > at hudson.FilePath.act(FilePath.java:998) > at hudson.FilePath.act(FilePath.java:987) > at hudson.FilePath.deleteRecursive(FilePath.java:1192) > at > org.jenkinsci.plugins.workflow.steps.DeleteDirStep$Execution.run(DeleteDirStep.java:77) > at > org.jenkinsci.plugins.workflow.steps.DeleteDirStep$Execution.run(DeleteDirStep.java:69) > at > org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:49) > at hudson.security.ACL.impersonate(ACL.java:290) > at > org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:46) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused: java.io.IOException: Unable to delete > 'C:\Jenkins2\workspace\Stan_develop-3CVGV42HAM7J3BLI3M44PCR6M52FN6ZQ65IEZ6TOSHJURSBO2PPA\src\test\test-models\bad\read_only'. > Tried 3 times (of a maximum of 3) waiting 0.1 sec between attempts. > at hudson.Util.deleteFile(Util.java:258) > at hudson.FilePath.deleteRecursive(FilePath.java:1225) > at hudson.FilePath.deleteContentsRecursive(FilePath.java:1234) > at hudson.FilePath.deleteRecursive(FilePath.java:1216) > at hudson.FilePath.deleteContentsRecursive(FilePath.java:1234) > at hudson.FilePath.deleteRecursive(FilePath.java:1216) > at hudson.FilePath.deleteContentsRecursive(FilePath.java:1234) > at hudson.FilePath.deleteRecursive(FilePath.java:1216) > at hudson.FilePath.deleteContentsRecursive(FilePath.java:1234) > at hudson.FilePath.deleteRecursive(FilePath.java:1216) > at hudson.FilePath.deleteContentsRecursive(FilePath.java:1234) > at hudson.FilePath.deleteRecursive(FilePath.java:1216) > at hudson.FilePath.access$1100(FilePath.java:208) > at hudson.FilePath$13.invoke(FilePath.java:1195) > at hudson.FilePath$13.invoke(FilePath.java:1192) > at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2816) > at hudson.remoting.UserRequest.perform(UserRequest.java:210) > at hudson.remoting.UserRequest.perform(UserRequest.java:53) > at hudson.remoting.Request$2.run(Request.java:364) > at > hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) > at java.util.concurrent.FutureTask.run(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at hudson.remoting.Engine$1$1.run(Engine.java:94) > at java.lang.Thread.run(Unknown Source) > Also: java.nio.file.AccessDeniedException: > C:\Jenkins2\workspace\Stan_develop-3CVGV42HAM7J3BLI3M44PCR6M52FN6ZQ65IEZ6TOSHJURSBO2PPA\src\test\test-models\bad\stanc_helper.stan > at sun.nio.fs.WindowsException.translateToIOException(Unknown Source) > at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) > at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source) > at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(Unknown Source) > at java.nio.file.spi.FileSystemProvider.newOutputStream(Unknown Source) > at java.nio.file.Files.newOutputStream(Unknown Source) > at hudson.util.IOUtils.copy(IOUtils.java:42) > at hudson.FilePath.readFromTar(FilePath.java:2363) > Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to > JNLP4-connect connection from > gelman-group-win.stat.columbia.edu/128.59.76.64:50229 > at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1737) > at hudson.remoting.UserResponse.retrieve(UserRequest.java:313) > at hudson.remoting.Channel.call(Channel.java:952) > at hudson.FilePath.act(FilePath.java:998) > at hudson.FilePath.act(FilePath.java:987) > at hudson.FilePath.untar(FilePath.java:540) > at > org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:129) > at > org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:74) > at > org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:61) > at > org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:49) > at hudson.security.ACL.impersonate(ACL.java:290) > at > org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused: java.io.IOException: Failed to extract StanSetup.tar.gz > at hudson.FilePath.readFromTar(FilePath.java:2373) > at hudson.FilePath.access$400(FilePath.java:208) > at hudson.FilePath$4.invoke(FilePath.java:542) > at hudson.FilePath$4.invoke(FilePath.java:540) > at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2816) > at hudson.remoting.UserRequest.perform(UserRequest.java:210) > at hudson.remoting.UserRequest.perform(UserRequest.java:53) > at hudson.remoting.Request$2.run(Request.java:364) > at > hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) > at java.util.concurrent.FutureTask.run(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at hudson.remoting.Engine$1$1.run(Engine.java:94) > at java.lang.Thread.run(Unknown Source) > Caused: java.io.IOException: remote file operation failed: > C:\Jenkins2\workspace\Stan_develop-3CVGV42HAM7J3BLI3M44PCR6M52FN6ZQ65IEZ6TOSHJURSBO2PPA > at hudson.remoting.Channel@47c318f0:JNLP4-connect connection from > gelman-group-win.stat.columbia.edu/128.59.76.64:50229 > at hudson.FilePath.act(FilePath.java:1005) > at hudson.FilePath.act(FilePath.java:987) > at hudson.FilePath.untar(FilePath.java:540) > at > org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:129) > at > org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:74) > at > org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:61) > Caused: java.io.IOException: remote file operation failed: > C:\Jenkins2\workspace\Stan_develop-3CVGV42HAM7J3BLI3M44PCR6M52FN6ZQ65IEZ6TOSHJURSBO2PPA > at hudson.remoting.Channel@47c318f0:JNLP4-connect connection from > gelman-group-win.stat.columbia.edu/128.59.76.64:50229 > at hudson.FilePath.act(FilePath.java:1005) > at hudson.FilePath.act(FilePath.java:987) > at hudson.FilePath.deleteRecursive(FilePath.java:1192) > at > org.jenkinsci.plugins.workflow.steps.DeleteDirStep$Execution.run(DeleteDirStep.java:77) > at > org.jenkinsci.plugins.workflow.steps.DeleteDirStep$Execution.run(DeleteDirStep.java:69) > at > org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:49) > at hudson.security.ACL.impersonate(ACL.java:290) > at > org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:46) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-users/6eaa7b8d-d69b-4c5e-a818-2a94643ce96d%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CABxGUTjt-yWgMg58S%3DQNV7wy9r84J4hr270rjqTU-ck0sps%2BpA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
