[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
Sandro Bonazzola created OVIRT-2814:
---
Summary: reposync fails syncing repos on completely unrelated
paths (concurrency issue?)
Key: OVIRT-2814
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: Sandro Bonazzola
Assignee: infra
About job
https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
As you can see below
05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
--repoid=centos-kvm-common-el7 --arch=x86_64
--cachedir=/home/jenkins/mirrors_cache
--download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
--norepopath --newest-only --urls --quiet
the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
using as cache directory "/home/jenkins/mirrors_cache"
but in "/home/jenkins/mirrors_cache" there's
"/home/jenkins/mirrors_cache/fedora-base-fc29":
05:35:16 Traceback (most recent call last):
05:35:16 File "/usr/bin/reposync", line 373, in
05:35:16 main()
05:35:16 File "/usr/bin/reposync", line 185, in main
05:35:16 my.doRepoSetup()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
681, in doRepoSetup
05:35:16 return self._getRepos(thisrepo, True)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
721, in _getRepos
05:35:16 self._repos.doSetup(thisrepo)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
in doSetup
05:35:16 self.retrieveAllMD()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
in retrieveAllMD
05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
1468, in _commonLoadRepoXML
05:35:16 local = self.cachedir + '/repomd.xml'
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
777, in
05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
760, in _dirGetAttr
05:35:16 self.dirSetup()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
738, in dirSetup
05:35:16 self._dirSetupMkdir_p(dir)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
715, in _dirSetupMkdir_p
05:35:16 raise Errors.RepoError, msg
05:35:16 yum.Errors.RepoError: Error making cache directory:
/home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
this looks like a concurrency issue, a lock should be used in order to
prevent two instances to use the same cache directory at the same time or
use separate cache directories for different repos.
--
Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100113)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/W4O2LZXGQTI6EW2EA5XHXTN23RWIYQ7A/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39909#comment-39909
]
Barak Korren commented on OVIRT-2814:
-
{code}
this looks like a concurrency issue, a lock should be used in order to
prevent two instances to use the same cache directory at the same time or
use separate cache directories for different repos.
{code}
As you can see in the path name - it is unique for the repo, and there is only
one update job per repo that does not run concurrently. So the last sentence
you wrote above holds.
This might be a cleanup issues, where some failure scenario leaves some lock
files behind.
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100113)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/WE5IYWTQQVIDGTUHUMAKRNLX7GT552RO/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39911#comment-39911
]
Eyal Edri commented on OVIRT-2814:
--
Is there anything we can/should do to fix this or next runs should work?
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100113)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/6H63JN5HBUHWZZOTTML4TZ55UU72J7IS/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39912#comment-39912
]
Barak Korren commented on OVIRT-2814:
-
ticket is from 5 days ago, last run seems to have passed:
https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2950/
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100113)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/QVJIPHKEFORATIJZEXZUOFOT42FFS4UU/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Marchukov updated OVIRT-2814:
---
Resolution: Fixed
Status: Done (was: To Do)
Working now, seems like a temporary issue.
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/HAPTTE2P3NGFE3Q7CR3TD6N46YQVMNJT/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39917#comment-39917
]
Sandro Bonazzola commented on OVIRT-2814:
-
correct me if I’m worng but no real fix has been included right? lock file has
just been removed and we’re going to see this happen again sooner or later.
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/5TRRWUNPPVVFV23JMO77UWYKBLLNIYMF/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39918#comment-39918
]
Barak Korren commented on OVIRT-2814:
-
{quote}
correct me if I’m worng but no real fix has been included right? lock file has
just been removed and we’re going to see this happen again sooner or later.
{quote}
Ok, I see whay you talk about now, I missed the fact that the failure was in a
job for a different mirror.
To solve this, we need to make sure that when we sync repo X, the {{reposync}}
command only sees the part of "{{mirrors-reposync.conf}}" that is relevant for
repo X. It should probably be doable by adding a {{sed}} command to
{{mirror_mgr.sh}} that will remove the unneeded repos from the file before
running {{reposync}}.
[~accountid:5b25ec3f8b08c009c48b25c9] this is exactly what we've talked about
before you left today...
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/YTD4KI6LRDOZPSYCSNYVGXNLTVYSSYF7/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barak Korren reopened OVIRT-2814:
-
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Components: CI Mirrors
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/2YS4H2RQ55P64QSGRCUJPLTCJCJ44CFP/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Barak Korren updated OVIRT-2814:
Components: CI Mirrors
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Components: CI Mirrors
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/GP27PBBAXR56I7POEAVIMGUBC75YCXW4/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39918#comment-39918
]
Barak Korren edited comment on OVIRT-2814 at 10/22/19 1:54 PM:
---
{quote}
correct me if I’m worng but no real fix has been included right? lock file has
just been removed and we’re going to see this happen again sooner or later.
{quote}
Ok, I see what you talk about now, I missed the fact that the failure was in a
job for a different mirror.
To solve this, we need to make sure that when we sync repo X, the {{reposync}}
command only sees the part of "{{mirrors-reposync.conf}}" that is relevant for
repo X. It should probably be doable by adding a {{sed}} command to
{{mirror_mgr.sh}} that will remove the unneeded repos from the file before
running {{reposync}}.
[~accountid:5b25ec3f8b08c009c48b25c9] this is exactly what we've talked about
before you left today...
was (Author: bkor...@redhat.com):
{quote}
correct me if I’m worng but no real fix has been included right? lock file has
just been removed and we’re going to see this happen again sooner or later.
{quote}
Ok, I see whay you talk about now, I missed the fact that the failure was in a
job for a different mirror.
To solve this, we need to make sure that when we sync repo X, the {{reposync}}
command only sees the part of "{{mirrors-reposync.conf}}" that is relevant for
repo X. It should probably be doable by adding a {{sed}} command to
{{mirror_mgr.sh}} that will remove the unneeded repos from the file before
running {{reposync}}.
[~accountid:5b25ec3f8b08c009c48b25c9] this is exactly what we've talked about
before you left today...
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Components: CI Mirrors
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Marchukov closed OVIRT-2814.
--
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Components: CI Mirrors
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/Z372NA5WWGT3ETO455HI72HNP7THW5MG/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Marchukov reopened OVIRT-2814:
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Components: CI Mirrors
>Reporter: Sandro Bonazzola
>Assignee: infra
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/SM7O6Q3L6VE75BW5TFSH72VY7N6VIXYG/
[JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
[
https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Marchukov reassigned OVIRT-2814:
--
Assignee: Evgheni Dereveanchin (was: infra)
> reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> ---
>
> Key: OVIRT-2814
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Components: CI Mirrors
>Reporter: Sandro Bonazzola
>Assignee: Evgheni Dereveanchin
>
> About job
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> As you can see below
> 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> --repoid=centos-kvm-common-el7 --arch=x86_64
> --cachedir=/home/jenkins/mirrors_cache
> --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> --norepopath --newest-only --urls --quiet
> the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> using as cache directory "/home/jenkins/mirrors_cache"
> but in "/home/jenkins/mirrors_cache" there's
> "/home/jenkins/mirrors_cache/fedora-base-fc29":
> 05:35:16 Traceback (most recent call last):
> 05:35:16 File "/usr/bin/reposync", line 373, in
> 05:35:16 main()
> 05:35:16 File "/usr/bin/reposync", line 185, in main
> 05:35:16 my.doRepoSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 681, in doRepoSetup
> 05:35:16 return self._getRepos(thisrepo, True)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> 721, in _getRepos
> 05:35:16 self._repos.doSetup(thisrepo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
> in doSetup
> 05:35:16 self.retrieveAllMD()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> in retrieveAllMD
> 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 1468, in _commonLoadRepoXML
> 05:35:16 local = self.cachedir + '/repomd.xml'
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 777, in
> 05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 760, in _dirGetAttr
> 05:35:16 self.dirSetup()
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 738, in dirSetup
> 05:35:16 self._dirSetupMkdir_p(dir)
> 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> 715, in _dirSetupMkdir_p
> 05:35:16 raise Errors.RepoError, msg
> 05:35:16 yum.Errors.RepoError: Error making cache directory:
> /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> --
> Sandro Bonazzola
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100114)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/infra@ovirt.org/message/G3TJOMP7BTFIULZURT75MAI4VEXKCLDA/
Re: [JIRA] (OVIRT-2814) reposync fails syncing repos on completely unrelated paths (concurrency issue?)
Is there anything we can/should do to fix this or next runs should work?
On Tue, Oct 22, 2019 at 9:50 AM Barak Korren (oVirt JIRA) <
j...@ovirt-jira.atlassian.net> wrote:
>
> [
> https://ovirt-jira.atlassian.net/browse/OVIRT-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=39909#comment-39909
> ]
>
> Barak Korren commented on OVIRT-2814:
> -
>
> {code}
> this looks like a concurrency issue, a lock should be used in order to
> prevent two instances to use the same cache directory at the same time or
> use separate cache directories for different repos.
> {code}
>
> As you can see in the path name - it is unique for the repo, and there is
> only one update job per repo that does not run concurrently. So the last
> sentence you wrote above holds.
>
> This might be a cleanup issues, where some failure scenario leaves some
> lock files behind.
>
> > reposync fails syncing repos on completely unrelated paths (concurrency
> issue?)
> >
> ---
> >
> > Key: OVIRT-2814
> > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2814
> > Project: oVirt - virtualization made easy
> > Issue Type: By-EMAIL
> >Reporter: Sandro Bonazzola
> >Assignee: infra
> >
> > About job
> >
> https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console
> > As you can see below
> > 05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
> > --repoid=centos-kvm-common-el7 --arch=x86_64
> > --cachedir=/home/jenkins/mirrors_cache
> > --download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
> > --norepopath --newest-only --urls --quiet
> > the sync is related to
> "/var/www/html/repos/yum/centos-kvm-common-el7/base"
> > using as cache directory "/home/jenkins/mirrors_cache"
> > but in "/home/jenkins/mirrors_cache" there's
> > "/home/jenkins/mirrors_cache/fedora-base-fc29":
> > 05:35:16 Traceback (most recent call last):
> > 05:35:16 File "/usr/bin/reposync", line 373, in
> > 05:35:16 main()
> > 05:35:16 File "/usr/bin/reposync", line 185, in main
> > 05:35:16 my.doRepoSetup()
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> > 681, in doRepoSetup
> > 05:35:16 return self._getRepos(thisrepo, True)
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/__init__.py", line
> > 721, in _getRepos
> > 05:35:16 self._repos.doSetup(thisrepo)
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line
> 157,
> > in doSetup
> > 05:35:16 self.retrieveAllMD()
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
> > in retrieveAllMD
> > 05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> > 1468, in _commonLoadRepoXML
> > 05:35:16 local = self.cachedir + '/repomd.xml'
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> > 777, in
> > 05:35:16 cachedir = property(lambda self:
> self._dirGetAttr('cachedir'))
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> > 760, in _dirGetAttr
> > 05:35:16 self.dirSetup()
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> > 738, in dirSetup
> > 05:35:16 self._dirSetupMkdir_p(dir)
> > 05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
> > 715, in _dirSetupMkdir_p
> > 05:35:16 raise Errors.RepoError, msg
> > 05:35:16 yum.Errors.RepoError: Error making cache directory:
> > /home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
> > exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'
> > this looks like a concurrency issue, a lock should be used in order to
> > prevent two instances to use the same cache directory at the same time or
> > use separate cache directories for different repos.
> > --
> > Sandro Bonazzola
>
>
>
> --
> This message was sent by Atlassian Jira
> (v1001.0.0-SNAPSHOT#100113)
> ___
> Infra mailing list -- infra@ovirt.org
> To unsubscribe send an email to infra-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/WE5IYWTQQVIDGTUHUMAKRNLX7GT552RO/
>
--
Eyal edri
He / Him / His
MANAGER
CONTINUOUS PRODUCTIZATION
SYSTEM ENGINEERING
Red Hat
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel)
___
Infra mailing list -- infra@ovirt.org
To unsubscribe send an email to infra-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Co
