[JIRA] (JENKINS-10905) Deleting previous project files regularly fails the build
pjdarton commented on JENKINS-10905 Deleting previous project files regularly fails the build I have a prospective fix for this in JENKINS-15331. However, on Windows XP, I've found that it is possible to workaround the issue by configuring any anti-virus to not scan the workspace, and to ensure that the windows search service is either completely disabled or configured to avoid files in the workspace. In my experience, it's only on Vista & Windows 7 onwards that it's impossible to disable this "malware-like" behavior as it's now too deeply embedded in the OS. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-10905) Deleting previous project files regularly fails the build
Paul Espinoza commented on JENKINS-10905 Deleting previous project files regularly fails the build I am also experiencing these occasional checkout failures while trying to delete the files from the last build. The Checkout settings are set to 'Always check out a fresh copy'. This is happening on a Windows XP (version 5.1) node connected to a Linux master Jenkins server. Jenkins version 1.466.2. I have experienced this in 7 of my last 64 builds (which are scheduled to run nightly), so it's happening roughly 10% of the time for me. Triggering the job again works as a temporary work around. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-10905) Deleting previous project files regularly fails the build
[ https://issues.jenkins-ci.org/browse/JENKINS-10905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=161719#comment-161719 ] nyoung02 commented on JENKINS-10905: This issue (raised by me) and JENKINS-3053 are both reported on Windows. However I also see the same issue on some Linux and Unix slaves on larger checkouts (3gb+), so I'm not sure the suggested resolutions such as search service/anti-virus are relevant to this issue - sure, it may help, and will undoubtedly help with overall speed of builds. A possible workaround would be to change the checkout strategy to 'update' and do the svn checkout into a directory under %BUILD_ID% or %BUILD_NUMBER% - then each checkout would be definitely in a clean directory. To avoid filling the hard drive up though with old builds, we'd also need to delete the workspace at the end of the build - I think there's a plugin to do that too, so possibly worth a try... > Deleting previous project files regularly fails the build > - > > Key: JENKINS-10905 > URL: https://issues.jenkins-ci.org/browse/JENKINS-10905 > Project: Jenkins > Issue Type: Bug > Components: core >Affects Versions: current > Environment: Window 7 operating in master/slave environment (the > failing machine is the slave) >Reporter: nyoung02 >Priority: Blocker > > I have a project that is quite large. I'm not sure if it's coincidental or > not, but since upgrading to 1.428, I've been getting a number of intermittent > build failures, that actually fail in the cleanup stage deleting files from > the previous build. My svn settings are set to 'clean checkout' although I > also get this problem if I select 'revert and update'. Is it possible to make > this stage more tolerant of locked files (perhaps by increasing the number of > retries?) > 09:51:40 Started by user niy > 09:51:40 Building remotely on nwb-r5win32b > 09:51:40 Cleaning workspace C:\jenkins\workspace\R5_Win_Overnight > 09:52:29 hudson.util.IOException2: remote file operation failed: > c:\jenkins\workspace\R5_Win_Overnight at > hudson.remoting.Channel@470d6c27:nwb-r5win32b > 09:52:29 at hudson.FilePath.act(FilePath.java:754) > 09:52:29 at hudson.FilePath.act(FilePath.java:740) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676) > 09:52:29 at > hudson.model.AbstractProject.checkout(AbstractProject.java:1193) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443) > 09:52:29 at hudson.model.Run.run(Run.java:1376) > 09:52:29 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) > 09:52:29 at > hudson.model.ResourceController.execute(ResourceController.java:88) > 09:52:29 at hudson.model.Executor.run(Executor.java:230) > 09:52:29 Caused by: java.io.IOException: Unable to delete > c:\jenkins\workspace\R5_Win_Overnight\viscob\coretech\checker\src\tools\.svn > - files in dir: > [c:\jenkins\workspace\R5_Win_Overnight\viscob\coretech\checker\src\tools\.svn\tmp] > 09:52:29 at hudson.Util.deleteFile(Util.java:265) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:316) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at > hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:74) > 09:52:29 at > hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:136) > 09:52:29 at > hudson.scm.SubversionSCM$CheckOutTask.perform(SubversionSCM.java:773) > 09:52:29 at > hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:754) > 09:52:29 at > hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:738) > 09:52:29 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1994) > 09:52:29 at hudson.remoting.UserRequest.perform(UserRequest.java:118) > 09:52:29 at hudson.remoting.UserRequest.perform(UserRequest.java:48) > 09:52:29
[JIRA] (JENKINS-10905) Deleting previous project files regularly fails the build
[ https://issues.jenkins-ci.org/browse/JENKINS-10905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=161682#comment-161682 ] pjdarton edited comment on JENKINS-10905 at 4/17/12 2:24 PM: - I'm glad it's not just me - I keep getting these as well. In my case, it's from the Accurev plugin calling hudson.Util.deleteContentsRecursive and getting spurious errors from the filesystem. I believe that the underlying cause is an operating system bug, either that or a very common misconfiguration, as I (sometimes) get the same problems doing "rd /s /q xxx" from a CMD window, and (sometimes) find that Explorer fails to delete everything when doing a Shift+Delete on directories. i.e. I think that Windows sometimes locks files briefly, causing deletes to fail. Note: I've configured the Windows Search service to leave well alone, and also the anti-virus, so I don't think it's them - I've had these problems from both of those on previous versions of Windows, but on Windows 7 it doesn't seem to be enough. However, regardless of the cause of the problem, the workaround is probably to delete everything that can be deleted, and to retry a number of times until either all files are deleted, or we've exhausted the retries. I wouldn't call it a blocker though - just triggering the build again usually makes the problem disappear. i.e. it's as annoying as hell, but it isn't a blocker. Note: JENKINS-3053 appears to be reporting the same problem. was (Author: pjdarton): I'm glad it's not just me - I keep getting these as well. In my case, it's from the Accurev plugin calling hudson.Util.deleteContentsRecursive and getting spurious errors from the filesystem. I believe that the underlying cause is an operating system bug, either that or a very common misconfiguration, as I (sometimes) get the same problems doing "rd /s /q xxx" from a CMD window, and (sometimes) find that Explorer fails to delete everything when doing a Shift+Delete on directories. i.e. I think that Windows sometimes locks files briefly, causing deletes to fail. Note: I've configured the Windows Search service to leave well alone, and also the anti-virus, so I don't think it's them - I've had these problems from both of those on previous versions of Windows, but on Windows 7 it doesn't seem to be enough. However, regardless of the cause of the problem, the workaround is probably to delete everything that can be deleted, and to retry a number of times until either all files are deleted, or we've exhausted the retries. I wouldn't call it a blocker though - just triggering the build again usually makes the problem disappear. i.e. it's as annoying as hell, but it isn't a blocker. Note: JENKINS-3052 appears to be reporting the same problem. > Deleting previous project files regularly fails the build > - > > Key: JENKINS-10905 > URL: https://issues.jenkins-ci.org/browse/JENKINS-10905 > Project: Jenkins > Issue Type: Bug > Components: core >Affects Versions: current > Environment: Window 7 operating in master/slave environment (the > failing machine is the slave) >Reporter: nyoung02 >Priority: Blocker > > I have a project that is quite large. I'm not sure if it's coincidental or > not, but since upgrading to 1.428, I've been getting a number of intermittent > build failures, that actually fail in the cleanup stage deleting files from > the previous build. My svn settings are set to 'clean checkout' although I > also get this problem if I select 'revert and update'. Is it possible to make > this stage more tolerant of locked files (perhaps by increasing the number of > retries?) > 09:51:40 Started by user niy > 09:51:40 Building remotely on nwb-r5win32b > 09:51:40 Cleaning workspace C:\jenkins\workspace\R5_Win_Overnight > 09:52:29 hudson.util.IOException2: remote file operation failed: > c:\jenkins\workspace\R5_Win_Overnight at > hudson.remoting.Channel@470d6c27:nwb-r5win32b > 09:52:29 at hudson.FilePath.act(FilePath.java:754) > 09:52:29 at hudson.FilePath.act(FilePath.java:740) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676) > 09:52:29 at > hudson.model.AbstractProject.checkout(AbstractProject.java:1193) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443) > 09:52:29 at hudson.model.Run.run(Run.java:1376) > 09:52:29 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) > 09:52:29 at > hudson.model.ResourceController.execute(ResourceController.java:88) >
[JIRA] (JENKINS-10905) Deleting previous project files regularly fails the build
[ https://issues.jenkins-ci.org/browse/JENKINS-10905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=161682#comment-161682 ] pjdarton edited comment on JENKINS-10905 at 4/17/12 2:23 PM: - I'm glad it's not just me - I keep getting these as well. In my case, it's from the Accurev plugin calling hudson.Util.deleteContentsRecursive and getting spurious errors from the filesystem. I believe that the underlying cause is an operating system bug, either that or a very common misconfiguration, as I (sometimes) get the same problems doing "rd /s /q xxx" from a CMD window, and (sometimes) find that Explorer fails to delete everything when doing a Shift+Delete on directories. i.e. I think that Windows sometimes locks files briefly, causing deletes to fail. Note: I've configured the Windows Search service to leave well alone, and also the anti-virus, so I don't think it's them - I've had these problems from both of those on previous versions of Windows, but on Windows 7 it doesn't seem to be enough. However, regardless of the cause of the problem, the workaround is probably to delete everything that can be deleted, and to retry a number of times until either all files are deleted, or we've exhausted the retries. I wouldn't call it a blocker though - just triggering the build again usually makes the problem disappear. i.e. it's as annoying as hell, but it isn't a blocker. Note: JENKINS-3052 appears to be reporting the same problem. was (Author: pjdarton): I'm glad it's not just me - I keep getting these as well. In my case, it's from the Accurev plugin calling hudson.Util.deleteContentsRecursive and getting spurious errors from the filesystem. I believe that the underlying cause is an operating system bug, either that or a very common misconfiguration, as I (sometimes) get the same problems doing "rd /s /q xxx" from a CMD window, and (sometimes) find that Explorer fails to delete everything when doing a Shift+Delete on directories. i.e. I think that Windows sometimes locks files briefly, causing deletes to fail. Note: I've configured the Windows Search service to leave well alone, and also the anti-virus, so I don't think it's them - I've had these problems from both of those on previous versions of Windows, but on Windows 7 it doesn't seem to be enough. However, regardless of the cause of the problem, the workaround is probably to delete everything that can be deleted, and to retry a number of times until either all files are deleted, or we've exhausted the retries. I wouldn't call it a blocker though - just triggering the build again usually makes the problem disappear. i.e. it's as annoying as hell, but it isn't a blocker. > Deleting previous project files regularly fails the build > - > > Key: JENKINS-10905 > URL: https://issues.jenkins-ci.org/browse/JENKINS-10905 > Project: Jenkins > Issue Type: Bug > Components: core >Affects Versions: current > Environment: Window 7 operating in master/slave environment (the > failing machine is the slave) >Reporter: nyoung02 >Priority: Blocker > > I have a project that is quite large. I'm not sure if it's coincidental or > not, but since upgrading to 1.428, I've been getting a number of intermittent > build failures, that actually fail in the cleanup stage deleting files from > the previous build. My svn settings are set to 'clean checkout' although I > also get this problem if I select 'revert and update'. Is it possible to make > this stage more tolerant of locked files (perhaps by increasing the number of > retries?) > 09:51:40 Started by user niy > 09:51:40 Building remotely on nwb-r5win32b > 09:51:40 Cleaning workspace C:\jenkins\workspace\R5_Win_Overnight > 09:52:29 hudson.util.IOException2: remote file operation failed: > c:\jenkins\workspace\R5_Win_Overnight at > hudson.remoting.Channel@470d6c27:nwb-r5win32b > 09:52:29 at hudson.FilePath.act(FilePath.java:754) > 09:52:29 at hudson.FilePath.act(FilePath.java:740) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676) > 09:52:29 at > hudson.model.AbstractProject.checkout(AbstractProject.java:1193) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443) > 09:52:29 at hudson.model.Run.run(Run.java:1376) > 09:52:29 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) > 09:52:29 at > hudson.model.ResourceController.execute(ResourceController.java:88) > 09:52:29 at hudson.model.Executor.run(Executor.java:230)
[JIRA] (JENKINS-10905) Deleting previous project files regularly fails the build
[ https://issues.jenkins-ci.org/browse/JENKINS-10905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=161682#comment-161682 ] pjdarton commented on JENKINS-10905: I'm glad it's not just me - I keep getting these as well. In my case, it's from the Accurev plugin calling hudson.Util.deleteContentsRecursive and getting spurious errors from the filesystem. I believe that the underlying cause is an operating system bug, either that or a very common misconfiguration, as I (sometimes) get the same problems doing "rd /s /q xxx" from a CMD window, and (sometimes) find that Explorer fails to delete everything when doing a Shift+Delete on directories. i.e. I think that Windows sometimes locks files briefly, causing deletes to fail. Note: I've configured the Windows Search service to leave well alone, and also the anti-virus, so I don't think it's them - I've had these problems from both of those on previous versions of Windows, but on Windows 7 it doesn't seem to be enough. However, regardless of the cause of the problem, the workaround is probably to delete everything that can be deleted, and to retry a number of times until either all files are deleted, or we've exhausted the retries. I wouldn't call it a blocker though - just triggering the build again usually makes the problem disappear. i.e. it's as annoying as hell, but it isn't a blocker. > Deleting previous project files regularly fails the build > - > > Key: JENKINS-10905 > URL: https://issues.jenkins-ci.org/browse/JENKINS-10905 > Project: Jenkins > Issue Type: Bug > Components: core >Affects Versions: current > Environment: Window 7 operating in master/slave environment (the > failing machine is the slave) >Reporter: nyoung02 >Priority: Blocker > > I have a project that is quite large. I'm not sure if it's coincidental or > not, but since upgrading to 1.428, I've been getting a number of intermittent > build failures, that actually fail in the cleanup stage deleting files from > the previous build. My svn settings are set to 'clean checkout' although I > also get this problem if I select 'revert and update'. Is it possible to make > this stage more tolerant of locked files (perhaps by increasing the number of > retries?) > 09:51:40 Started by user niy > 09:51:40 Building remotely on nwb-r5win32b > 09:51:40 Cleaning workspace C:\jenkins\workspace\R5_Win_Overnight > 09:52:29 hudson.util.IOException2: remote file operation failed: > c:\jenkins\workspace\R5_Win_Overnight at > hudson.remoting.Channel@470d6c27:nwb-r5win32b > 09:52:29 at hudson.FilePath.act(FilePath.java:754) > 09:52:29 at hudson.FilePath.act(FilePath.java:740) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:731) > 09:52:29 at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:676) > 09:52:29 at > hudson.model.AbstractProject.checkout(AbstractProject.java:1193) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:555) > 09:52:29 at > hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:443) > 09:52:29 at hudson.model.Run.run(Run.java:1376) > 09:52:29 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) > 09:52:29 at > hudson.model.ResourceController.execute(ResourceController.java:88) > 09:52:29 at hudson.model.Executor.run(Executor.java:230) > 09:52:29 Caused by: java.io.IOException: Unable to delete > c:\jenkins\workspace\R5_Win_Overnight\viscob\coretech\checker\src\tools\.svn > - files in dir: > [c:\jenkins\workspace\R5_Win_Overnight\viscob\coretech\checker\src\tools\.svn\tmp] > 09:52:29 at hudson.Util.deleteFile(Util.java:265) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:316) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at hudson.Util.deleteRecursive(Util.java:307) > 09:52:29 at hudson.Util.deleteContentsRecursive(Util.java:227) > 09:52:29 at > hudson.scm.subversion.CheckoutUpdater$1.perform(CheckoutUpdater.java:74) > 09:52:29 at > hudson.scm.subversion.WorkspaceUpdater$UpdateTask.delegateTo(WorkspaceUpdater.java:136) > 09:52:29 at > hudson.scm.SubversionSCM$CheckOutTask.perform(Subve
