Re: X509 login patches
On 12/14/2009 02:03 PM, Christos Triantafyllidis wrote: > Hi all and welcome me to the list :), Welcome, and thanks for the patches! Comments in-line. > i'm using koji since a few week and i needed X509 authentication. > Unfortunately current support for x509 was limited to: > a) Use of the CN part only from the subject DN as the username > Although traditionally CN can be the "username" of the user there are > cases (like in our PKI) where CN is just "Christos Triantafyllidis" and > of course many users can have the same name but different DNs. To avoid > this but also keep the backwards compatibility i have introduced a new > variable to be exported by both apache config (for git-web) and hub.conf > (for the rest of the tools) called EnvVarForUserName which defines which > variable to use as Username. For my case i have "EnvVarForUserName = > SSL_CLIENT_S_DN" which uses the whole DN as username. koji-hub already supports a DNUsernameComponent option. Rather than introduce a new config option, I think I'd rather see "DNUsernameComponent=DN" special-cased to mean "use the whole DN". I don't see any env. vars other than DN that would be useful for authentication. > b) Keep asking the user to provide their pass-phrase many times for the > the same operation > This leads (IMHO) many users to use password-less certificates. > Unfortunately this is not acceptable according to our PKI policy so i > added a callback to cache the passphrase within each koji execution. This looks very interesting, thanks. I'll see about testing it locally and merging it. I wonder if this could be extended to integrate with gnome-keyring (or similar) to provide once-per-session login for SSL certificates. I'll look into this. > I have created some patches to both this limitations and i have > uploaded the to my git repository[1]. Feel free to use/clone them. > > Best regards, > Christos Triantafyllidis > > [1] http://git.afroditi.hellasgrid.gr/git/grid.auth.gr/koji.git > > > > -- > Fedora-buildsys-list mailing list > Fedora-buildsys-list@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Postgresql Database Error
On 11/18/2009 02:15 AM, peng chen wrote: > hello, fedora-buildsys-list: > >when I requset a build task for pakcage "anaconda" to koji, > one errie error come out. > It detailed as follow: > pg.DatabaseError: error ' ERROR: new row for relation "task" violates > check constraint "task_weight_check" ' > in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' > I'm sure that the source rpm of anaconda is OK,because I succed to build > it in local mock environment. and > the repo is the same as the koji build server. > hope you do me a favor sincerely! Does one of the builds have a completion_time earlier than its create_event time? Koji uses the build duration to dynamically calculate the weight of the task. It should probably be checking for a negative result, but it doesn't. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: latest python-cheetah breaks koji.
On 09/30/2009 04:05 AM, Steve Traylen wrote: > Hi, > I would just submit a bug but unsure to which package to submit in > the first instance. > > On FC11 with > Hi > > With, > > koji-hub-1.3.1-2.fc11.noarch > koji-1.3.1-2.fc11.noarch > koji-builder-1.3.1-2.fc11.noarch > koji-web-1.3.1-2.fc11.noarch > koji-utils-1.3.1-2.fc11.noarch > python-cheetah-2.0.1-5.fc11.x86_64 > > then everything is good. > > Updating only python-cheetah to > > python-cheetah-2.2.2-1.fc11.x86_64 > > then python-web dumps to the browser as below, full traceback attached. > Downgrading back to > python-cheetah-2.0.1-5.fc11.x86_64 > > then all is good. > > File "/usr/lib64/python2.6/site-packages/Cheetah/Compiler.py", line > 1588, in __init__ >source = unicode(source) > > UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position > 374: ordinal not in range(128) > > full back trace attached. Sorry about that, my fault. I hadn't finished testing Koji with the new Cheetah before I pushed it. This patch should fix the errors: http://mikeb.fedorapeople.org/0001-handle-all-strings-as-unicode-for-compatibility-with.patch It'll be pushed to the Koji git repo shortly. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Về: Fedora-buildsys-list Digest, Vol 55 , Issue 2
On 09/06/2009 07:34 AM, NGUYEN VAN TAN wrote:
Thank you, I saw kojid log, but I don't understand what it mean.
2009-09-06 14:18:54,311 [INFO] koji.build.TaskManager: open task: {'waiting':
True, 'id': 7, 'weight': 0.10001}
2009-09-06 14:18:54,398 [WARNING] koji.build.TaskManager: TRACEBACK: Traceback
(most recent call last):
File "/usr/sbin/kojid", line 1275, in runTask
response = (handler.run(),)
File "/usr/sbin/kojid", line 1351, in run
return self.handler(*self.params,**self.opts)
File "/usr/sbin/kojid", line 2560, in handler
for f in os.listdir(self.datadir):
OSError: [Errno 2] No such file or directory:
'/tmp/koji/tasks/8/8/repo/repodata'
2009-09-06 14:18:54,450 [WARNING] koji.build.TaskManager: TRACEBACK: Traceback
(most recent call last):
File "/usr/sbin/kojid", line 1275, in runTask
response = (handler.run(),)
File "/usr/sbin/kojid", line 1351, in run
return self.handler(*self.params,**self.opts)
File "/usr/sbin/kojid", line 2560, in handler
for f in os.listdir(self.datadir):
OSError: [Errno 2] No such file or directory:
'/tmp/koji/tasks/9/9/repo/repodata'
Do you have any packages or external repos associated with the tag
you're trying to create a repo for? If not, then createrepo will never
get called, and it won't create the repodata/ subdirectory. Add a
package or external repo to the tag, and newRepo/createrepo tasks should
start succeeding.
2009-09-06 14:19:09,407 [INFO] koji.build.TaskManager: pids: {7: 10867, 8:
10873, 9: 10877}
2009-09-06 14:19:09,417 [INFO] koji.build.TaskManager: open task: {'waiting':
True, 'id': 7, 'weight': 0.10001, 'alert': True}
2009-09-06 14:19:09,418 [INFO] koji.build.TaskManager: Waking up task:
{'waiting': True, 'id': 7, 'weight': 0.10001, 'alert': True}
2009-09-06 14:19:09,424 [INFO] koji.build.TaskManager: Task 8 (pid 10873)
exited with status 0
2009-09-06 14:19:09,582 [WARNING] koji.build.TaskManager: FAULT:
Traceback (most recent call last):
File "/usr/sbin/kojid", line 1275, in runTask
response = (handler.run(),)
File "/usr/sbin/kojid", line 1351, in run
return self.handler(*self.params,**self.opts)
File "/usr/sbin/kojid", line 2517, in handler
results = self.wait(subtasks.values(), all=True, failany=True)
File "/usr/sbin/kojid", line 1438, in wait
return dict(session.host.taskWaitResults(self.id,subtasks))
File "/usr/lib/python2.5/site-packages/koji/__init__.py", line 1255, in
__call__
return self.__func(self.__name,args,opts)
File "/usr/lib/python2.5/site-packages/koji/__init__.py", line 1501, in
_callMethod
raise err
Fault:
2009-09-06 14:19:09,610 [INFO] koji.build.TaskManager: Expiring subsession 27
(task 8)
2009-09-06 14:19:09,666 [INFO] koji.build.TaskManager: Task 9 (pid 10877)
exited with status 0
2009-09-06 14:19:09,694 [INFO] koji.build.TaskManager: Expiring subsession 28
(task 9)
2009-09-06 14:19:25,813 [INFO] koji.build.TaskManager: pids: {7: 10867}
2009-09-06 14:19:25,943 [INFO] koji.build.TaskManager: Task 7 (pid 10867)
exited with status 0
2009-09-06 14:19:25,973 [INFO] koji.build.TaskManager: Expiring subsession 26
(task 7)
I also saw kojira log, but I can't find the what problem is.
2009-09-06 13:09:23,652 [INFO] koji.repo.manager: Found repo 24, state=INIT
2009-09-06 13:10:07,760 [INFO] koji.repo.manager: Problem: newRepo task 116 for
tag 2 is FAILED
2009-09-06 13:12:19,724 [INFO] koji.repo.manager: Created newRepo task 121 for
tag 2 (dist-foo-build)
2009-09-06 13:12:25,373 [INFO] koji.repo.manager: Found repo 25, state=INIT
2009-09-06 13:12:32,349 [INFO] koji.repo.manager: Problem: newRepo task 121 for
tag 2 is FAILED
2009-09-06 14:07:52,959 [INFO] koji: Entering main loop
2009-09-06 14:14:26,191 [INFO] koji.repo.manager: Created newRepo task 1 for
tag 2 (dist-foo-build)
2009-09-06 14:14:36,261 [INFO] koji.repo.manager: Found repo 1, state=INIT
2009-09-06 14:15:06,624 [INFO] koji.repo.manager: Problem: newRepo task 1 for
tag 2 is FAILED
2009-09-06 14:15:06,654 [INFO] koji.repo.manager: Created newRepo task 4 for
tag 2 (dist-foo-build)
2009-09-06 14:15:21,807 [INFO] koji.repo.manager: Problem: newRepo task 4 for
tag 2 is FAILED
2009-09-06 14:15:21,868 [INFO] koji.repo.manager: Found repo 2, state=INIT
2009-09-06 14:18:24,378 [INFO] koji.repo.manager: Created newRepo task 7 for
tag 2 (dist-foo-build)
2009-09-06 14:18:43,487 [INFO] koji.repo.manager: Found repo 3, state=INIT
2009-09-06 14:19:11,220 [INFO] koji.repo.manager: Problem: newRepo task 7 for
tag 2 is FAILED
2009-09-06 14:21:41,670 [INFO] koji.repo.manager: Created newRepo task 10 for
tag 2 (dist-foo-build)
2009-09-06 14:22:01,882 [INFO] koji.repo.manager: Found repo 4, state=INIT
2009-09-06 14:22:17,139 [INFO] koji.repo.manager: Problem: newRepo task 10 for
tag 2 is FAILED
I checked if /tmp/koji/tasks/8/8/repo/repodata exist and this folder is not
exist. Do you know this folder would be created by what kin
Re: Delete a build from a tag, How to do ?
Jitesh Shah wrote:
Hi,
On Thu, 2009-06-18 at 20:54 -0700, Jian Lee wrote:
Hello, everyone
I want to delete a build from a tag, How should I do ?
example, the tag tms2.0 have following build:
-
b43-fwcutter-011-5.tms2 michaelw 2009-06-19 09:52:18 complete
=
I first try to delete the build from tms2.0 tag by following cmd:
# koji call deleteBuild b43-fwcutter-011-5.tms2 tms2.0
GenericError: Cannot delete build, tagged: [{'id': 10, 'name': 'tms2.0'}]
==
Did a use a wrong api ? or did I can delete a build from a tag use
kojihub api ? Must to direct do on postgresql ?
deleteBuild will still keep some data in the database. If the reason for
deleting the build is so that you can build/import it again, then try
using "resetBuild". It will remove all the files and rpm data and mark
the build as "cancelled".
(Although a quick looks suggests that it removes entries from buildroot
too. So, use with care)
Do correct me if I am wrong.
You're correct, resetBuild is a *very* big hammer, only to be used in
exceptional circumstances (I can probably count the number of times I've
used it on one hand). deleteBuild is also only meant for use by the
koji-gc script, and should really never be called manually. That's why
neither of these methods are exposed via the koji cmd-line. If you're
going to use them, think very hard about what you're doing, why you're
doing it, and proceed with caution.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: NewRepo headache
Michael Schwendt wrote:
$ koji latest-pkg dist-f12 mcs
Build Tag Built by
mcs-0.7.1-4.fc12 dist-f12 mschwendt
$ koji latest-pkg dist-f12-build mcs
Build Tag Built by
mcs-0.7.1-4.fc12 dist-f12 mschwendt
Additionally, I've waited for the NewRepo task to complete.
And still the next build fails due to an unresolvable dep as it only
is seeing the previous mcs package release. Is that due to mock related
caches or what?
How to _reliably_ wait for a new package to become available in
the buildroot?
koji wait-repo --target dist-f12 --build mcs-0.7.1-4.fc12
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: lower N-V-R used in buildroot
Dan Horák wrote: Hi, in koji for Fedora/s390x I have 2 builds for wxGTK package https://s390.koji.fedoraproject.org/koji/packageinfo?packageID=7367 wxGTK-2.8.9-4.fc11 wxGTK-2.8.10-1.fc11 the higher NVR build is older but the lower NVR is used in the buildroot used to build a package (like wxPython) that requires wxGTK https://s390.koji.fedoraproject.org/koji/buildrootinfo?buildrootID=32535 https://s390.koji.fedoraproject.org/koji/taskinfo?taskID=74417 Could someone explain me what's wrong? Koji uses the "latest" package in a tag to populate the repodata/buildroots, where "latest" equals "most recently tagged". $ koji --server http://s390.koji.fedoraproject.org/kojihub list-tag-history --tag dist-f11 --package wxGTK Sun Apr 5 11:52:54 2009: Tagged wxGTK-2.8.10-1.fc11 with dist-f11 [still active] Thu Apr 9 19:08:22 2009: Tagged wxGTK-2.8.9-4.fc11 with dist-f11 [still active] wxGTK-2.8.9-4.fc11 was tagged most recently, so it's considered the "latest". To fix the issue, either untag wxGTK-2.8.9-4.fc11 or untag-and-retag wxGTK-2.8.10-1.fc11 (which would then become the "latest"). This behavior is intentional and useful, but it can cause unexpected behavior like this if you don't realize how Koji populates buildroots. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: How to remove the large directory after build ?
李建 wrote: Hello,everybody! My koji server has setup ok. but the mock dir is more and more large. How to remove it ? I mean if koji has this feature,I doesn't to do it myself. Is anyone can help me ? my mock directory info : 651Mgtes11.2-build-10-9 18M gtes11.2-build-13-10 5.6Mgtes11.2-build-15-15 549Mgtes11.2-build-1-6 5.5Mgtes11.2-build-16-15 18M gtes11.2-build-18-18 18M gtes11.2-build-19-18 579Mgtes11.2-build-21-20 560Mgtes11.2-build-22-20 3.5Ggtes11.2-build-23-26 548Mgtes11.2-build-2-6 567Mgtes11.2-build-3-6 583Mgtes11.2-build-5-9 562Mgtes11.2-build-6-9 670Mgtes11.2-build-9-9 6.2Mgtes11.3-build-24-28 6.1Mgtes11.3-build-25-28 637Mgtes11.3-build-26-30 609Mgtes11.3-build-28-30 3.5Ggtes11.3-build-30-30 3.0Ggtes11.3-build-31-30 1.8Ggtes11.3-build-32-32 1.8Ggtes11.3-build-34-32 583Mgtes11.3-build-35-33 562Mgtes11.3-build-37-33 888Mgtes11.3-build-39-34 852Mgtes11.3-build-40-34 408Mmoblin2-build-41-38 417Mmoblin2-build-42-39 == kojid manages the cleanup up the /var/lib/mock directory for builds it runs. Buildroots for successful builds will be cleaned up almost immediately. Buildroots for unsuccessful builds will be cleaned up after 4 hours, to give you a chance to debug the failure or copy relevant data out of the buildroot for later analysis. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Which RPM does mergerepo prefer?
Jitesh Shah wrote: Hi, I have attached an external repo to a build tag (Just trying out some hand-built RPMs). Now, if an RPM is present *both* in the build tag as well as the external repo, which one does mergerepos pick? It picks the rpm in the build tag (the local rpm). Is the decision based on version comparison (the greater version wins), or does the one in the build tag always get preference over the external repo (irrespective of the version)? The one in the build tag always wins. This is consistent with normal Koji tag inheritance, which is a depth-first, first-match-wins search. Koji never compares NVRs. If it is the latter, how do I tell mergerepos to prefer external repo over the build tag? (or atleast use version comparisons rather than preferring build tag) If you want to use the version from the external repo, untag any builds of that package from the local tag and let the external repo version be picked up as the first match. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: admin permission required
Tom Stage wrote:
Hi all
Well I have to admit that I am in the same boat as the last thread with the
same Subject, this config is also with SSL configured and it seems to be ok
and running good, I can log in to the web interface.
I have Koji installed on a Fedora 10 x86_64 system, and I have followed the
HowTo at http://fedoraproject.org/wiki/Koji/ServerHowTo and I to cant seem
to execute the following commands as an example:
System info:
Uname -a
Linux koji 2.6.27.21-170.2.56.fc10.x86_64 #1 SMP Mon Mar 23 23:08:10 EDT
2009 x86_64 x86_64 x86_64 GNU/Linux
Rpm -qa
koji-builder-1.3.1-1.fc10.noarch
koji-utils-1.3.1-1.fc10.noarch
koji-1.3.1-1.fc10.noarch
koji-web-1.3.1-1.fc10.noarch
koji-hub-1.3.1-1.fc10.noarch
SSL certificates created after the instructions in the howto, with one
exception, since this is a single host installation I have only created 3
types of certificates. The 1st one is the signing certificate. The 2nd one
is the certificates for the host, and used by all the koji services. The 3rd
one is the user certificates.
[r...@koji ~]koji call getLoggedInUser
{'id': 1, 'krb_principal': None, 'name': 'admin', 'status': 0, 'usertype':
0}
[r...@koji ~]koji add-user kojira
ActionNotAllowed: admin permission required
[r...@koji ~]koji add-host koji.dvos.dk x86_64
ActionNotAllowed: admin permission required
My users in the users table in postgres looks like this:
Id namepasswordstatus usertypekrb_principal
1 admin 0 0
2 koji.dvos.dk0 1
My permissions table looks like this:
Id name
1 admin
2 build
3 repo
I am confused and don't understand what I am doing wrong, and I am willing
to post my configuration files as well.
Any help is appreciated.
To grant a permission to a user you need to insert into the "user_perms"
table. The user_id column references the id of the users table, and the
perm_id references the id of the permissions table. In your case,
granting the "admin" user the "admin" permission would be accomplished
by running:
insert into user_perms (user_id, perm_id) values (1, 1);
After that, you can grant other permissions by using the "koji
grant-permission" command.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: One build for multiple platforms?
Steve Traylen wrote:
On Thu, May 14, 2009 at 8:08 PM, Mike Bonnet wrote:
Steve Traylen wrote:
Hi,
I thinking that the answer is that it is not currently possible but is
there any arrangement of configuration
to allow a build on say centos4 and centos5 concurrently.
A build tartget that has a fork to two buildroots and destination
tags. Both would need to work
for the overall task to work.
If you mean one "koji build" results in two builds being created in Koji,
then no that is not currently possible.
This sounds like something that could easily be handled with a Makefile
target though.
Create your separate dist-centos4 and dist-centos5 build/dest tags and
targets. Have a "make build" generate the appropriate SCM URL and call:
koji build --nowait dist-centos4 $SCMURL
koji build --nowait dist-centos5 $SCMURL
This works of course. I was hoping to force the dists to stay
in step though. builds are only tagged for el5 and el4 or not at all.
Rather like currently both i386 and x86_64 must both work to get either.
I can force the submitters to build on fc10 as well el4 and 5 to future proof
ourself.
This could be accomplished with "koji build --skip-tag". Then "koji
watch-task" on the two taskIDs, and if that returns 0 you can then "koji
tag-pkg" the two builds into their dest tags. It would probably be more
robust to code this up by using the xmlrpc api directly, so you can
check task success/failure and get the build NVR from the task ID
directly, rather than parsing the output of "koji taskinfo".
Assuming you're using %{?dist} in your specfiles and have the
buildsys-macros defined correctly (and uniquely) in the -build tags, this
will create 2 separate builds in different tags from the same sources.
An item relevant to this. When building say a -el4.src.rpm on el5
koji build dist-centos5 foobar-1.5.2.el4.src.rpm
will always fail because the resulting foobar-1.5.2.el5.src.rpm does
not name match foobar-1.5.2.el4.src.rpm
Is this check useful? It requires one to create to src.rpms before they can
be submitted to dist-centos4 and 5. I can't for instance just grab a
src.rpm package
from fc10 and submit it.
We only do this check for real (non --scratch) builds, and yes it is
useful. The NVR of the "build" object in the database comes from that
srpm. It would be very confusing if a build called "foo-1.0-1.el4"
generated an rpm named "foo-1.0-1.el5".
(a cheeky p.s, did you get the slides I sent. no comment is just fine)
Yes, haven't had a change to look them over yet though, I'll do that today.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: One build for multiple platforms?
Steve Traylen wrote:
Hi,
I thinking that the answer is that it is not currently possible but is
there any arrangement of configuration
to allow a build on say centos4 and centos5 concurrently.
A build tartget that has a fork to two buildroots and destination
tags. Both would need to work
for the overall task to work.
If you mean one "koji build" results in two builds being created in
Koji, then no that is not currently possible.
This sounds like something that could easily be handled with a Makefile
target though.
Create your separate dist-centos4 and dist-centos5 build/dest tags and
targets. Have a "make build" generate the appropriate SCM URL and call:
koji build --nowait dist-centos4 $SCMURL
koji build --nowait dist-centos5 $SCMURL
Assuming you're using %{?dist} in your specfiles and have the
buildsys-macros defined correctly (and uniquely) in the -build tags,
this will create 2 separate builds in different tags from the same sources.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: policy violation - but why?
Steve Traylen wrote: On Fri, May 8, 2009 at 4:04 PM, Mike Bonnet wrote: Steve Traylen wrote: On Fri, May 8, 2009 at 11:52 AM, Steve Traylen wrote: Hi, I was reliably building on a tag before but now receive ActionNotAllowed: policy violation A little bit more. ActionNotAllowed: policy violation -- all :: deny What version of Koji are you running? I believe there was a bug in earlier versions that caused a missing "policy" entry in /etc/koji-hub/hub.conf to result in denials of everything. That should be fixed in 1.3.1. Alternately you could add a policy entry to allow building from srpm into the dist-centos4 tag: [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: deny Hi Mike, I'm running stock FC10. koji-hub-1.3.1-1.fc10.noarch koji-1.3.1-1.fc10.noarch koji-web-1.3.1-1.fc10.noarch koji-utils-1.3.1-1.fc10.noarch koji-builder-1.3.1-1.fc10.noarch I've not had had any [policy] in the hub.conf file up to now and things have been okay, i.e I could build from cvs and svn for instance which I did yesterday. It's just building from srpm that has been blocked but I have not tried that in a while so that may have been the before the recent upgrade. Actually I was wrong, if no policy is present the default policy forbids building from srpm by anyone but an admin (this is for consistency with previous versions of Koji). But this is now overrideable with custom policy, whereas it was hard-coded before. Certainly making a very open policy [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: all and then things proceed, I'll tune that now. More generally now about policies. Do you have some description on these and what can be set? If nothing exists if you can give me something brief then I'll try and write something up for the wiki. Unfortunately the policy stuff is not well-documented at the moment, but we're working on fixing that. It is actually a very powerful mechanism that allows you to control what types of source repositories you can build from, setup elaborate building and tagging policy, etc. We'll get some basic documentation onto the fedoraproject.org wiki soon, and any help expanding on it at that point would be appreciated. As it happens I'm giving a presentation in a weeks time to my colleagues on mock, koji and mash and certainly any content (e.g diagrams) I produce I'll write up in a generic way for inclusion in documentation. That'd be great! Thanks again Steve Steve and can't seem shake it or understand why for a particular package Is is possible to get an explanation? http://skoji.cern.ch/koji/taskinfo?taskID=2918 This is following a build as CN=straylen koji build --nowait dist-centos4 ../SRPMS/mpich-1.2.7p1-2.el4.src.rpm My permissions. id | name | password | status | usertype | krb_principal +--+--++--+--- 1 | straylen | | 0 |0 | and user_id=1 does not appear in user_perms . i.e I am a boring user. The package has been added. koji list-pkgs --tag=dist-centos4 --package=mpich Package Tag Extra Arches Owner --- --- --- mpich dist-centos4 straylen Thanks again for the help. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: policy violation - but why?
Steve Traylen wrote: On Fri, May 8, 2009 at 11:52 AM, Steve Traylen wrote: Hi, I was reliably building on a tag before but now receive ActionNotAllowed: policy violation A little bit more. ActionNotAllowed: policy violation -- all :: deny What version of Koji are you running? I believe there was a bug in earlier versions that caused a missing "policy" entry in /etc/koji-hub/hub.conf to result in denials of everything. That should be fixed in 1.3.1. Alternately you could add a policy entry to allow building from srpm into the dist-centos4 tag: [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: deny Steve and can't seem shake it or understand why for a particular package Is is possible to get an explanation? http://skoji.cern.ch/koji/taskinfo?taskID=2918 This is following a build as CN=straylen koji build --nowait dist-centos4 ../SRPMS/mpich-1.2.7p1-2.el4.src.rpm My permissions. id | name | password | status | usertype | krb_principal +--+--++--+--- 1 | straylen | | 0 |0 | and user_id=1 does not appear in user_perms . i.e I am a boring user. The package has been added. koji list-pkgs --tag=dist-centos4 --package=mpich Package Tag Extra Arches Owner --- --- --- mpich dist-centos4 straylen Thanks again for the help. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Make sources?
Brian Kosick wrote: BTW my install/upgrade steps are abbreviated but not by much, I just had to go through the conf files created as rpmnew and merge the changes. There weren't that many... So I guess another question is: What's with /etc/koji-shadow and /etc/koji-gc? new features/daemons? Yes. koji-shadow duplicates the builds from an upstream Koji instance into your local Koji instance. koji-gc is a configurable garbage-collector for builds that have been obsoleted. Both are provided by the koji-utils subpackage. Information and details on the new features seems to be pretty sparse. At least I can't find much. If such docs exist, can someone point me to them? Yes, the documentation could definitely be better. Most of the relevant information is available on the fedoraproject.org wiki, see the ServerHowTo and ServerBootstrap pages. Feel free to make additions/improvements to those pages. Brian On Fri, 2009-05-01 at 19:07 -0600, Brian Kosick wrote: Hi All, So I think that I have successfully upgraded koji from 1.2.6 to 1.3.1 by doing this. I just have a few issues to work out 1) install the koji 1.3.1 rpms 2) update the db schema pgsql -h kojihost koji koji < /usr/share/doc/koji-1.3.1/docs/schema-upgrade-1.2-1.3.sql 3) started kojid and kojira When I did my first build, I got stuck with koji not having a mock group srpm-build I setup the mock group with koji add-group dist-el5-build srpm-build koji add-group-pkg dist-el5-build srpm-build pkg1 pkg2 pk3 I then added that group to my build dist with koji call addGroupList dist-mxl-el5-build srpm-build When I do a koji list-groups dist-mxl-el5-build i get koji list-groups dist-mxl-el5-build build [dist-mxl-el5-build] srpm-build [dist-el5-build] automake: None, default [dist-el5-build] bash: None, default [dist-el5-build] buildsys-macros: None, default [dist-el5-build] bzip2: None, default [dist-el5-build] bzip2-devel: None, default [dist-el5-build] coreutils: None, default [dist-el5-build] cpio: None, default [dist-el5-build] diffutils: None, default [dist-el5-build] elfutils: None, default [dist-el5-build] elfutils-libelf: None, default [dist-el5-build] file: None, default [dist-el5-build] gcc: None, default [dist-el5-build] gcc-c++: None, default [dist-el5-build] glibc: None, default [dist-el5-build] glibc-common: None, default [dist-el5-build] glibc-devel: None, default [dist-el5-build] glibc-headers: None, default [dist-el5-build] gzip: None, default [dist-el5-build] info: None, default [dist-el5-build] libselinux: None, default [dist-el5-build] libsemanage: None, default [dist-el5-build] libsepol: None, default [dist-el5-build] libtool-ltdl: None, default [dist-el5-build] make: None, default [dist-el5-build] patch: None, default [dist-el5-build] perl: None, default [dist-el5-build] policycoreutils: None, default [dist-el5-build] python: None, default [dist-el5-build] readline: None, default [dist-el5-build] readline-devel: None, default [dist-el5-build] redhat-release: None, default [dist-el5-build] redhat-rpm-config: None, default [dist-el5-build] rpm-build: None, default [dist-el5-build] rpm-libs: None, default [dist-el5-build] sed: None, default [dist-el5-build] selinux-policy: None, default [dist-el5-build] shadow-utils: None, default [dist-el5-build] sqlite: None, default [dist-el5-build] tar: None, default [dist-el5-build] unzip: None, default [dist-el5-build] which: None, default [dist-el5-build] zip: None, default [dist-el5-build] zlib-devel: None, default [dist-el5-build] It appears that the repos have regenned and now when I try to do a build, I'm getting DEBUG util.py:280: Executing command: ['make', 'sources'] DEBUG util.py:256: make: *** No rule to make target `sources'. Stop. DEBUG util.py:319: Child returncode was: 2 Has a new make command "sources" been created similar to "make srpm"? If so what does it do and what does koji expect back? Brian -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Make sources?
Brian Kosick wrote: Hi All, So I think that I have successfully upgraded koji from 1.2.6 to 1.3.1 by doing this. I just have a few issues to work out 1) install the koji 1.3.1 rpms 2) update the db schema pgsql -h kojihost koji koji < /usr/share/doc/koji-1.3.1/docs/schema-upgrade-1.2-1.3.sql 3) started kojid and kojira When I did my first build, I got stuck with koji not having a mock group srpm-build I setup the mock group with koji add-group dist-el5-build srpm-build koji add-group-pkg dist-el5-build srpm-build pkg1 pkg2 pk3 I then added that group to my build dist with koji call addGroupList dist-mxl-el5-build srpm-build When I do a koji list-groups dist-mxl-el5-build i get koji list-groups dist-mxl-el5-build build [dist-mxl-el5-build] srpm-build [dist-el5-build] automake: None, default [dist-el5-build] bash: None, default [dist-el5-build] buildsys-macros: None, default [dist-el5-build] bzip2: None, default [dist-el5-build] bzip2-devel: None, default [dist-el5-build] coreutils: None, default [dist-el5-build] cpio: None, default [dist-el5-build] diffutils: None, default [dist-el5-build] elfutils: None, default [dist-el5-build] elfutils-libelf: None, default [dist-el5-build] file: None, default [dist-el5-build] gcc: None, default [dist-el5-build] gcc-c++: None, default [dist-el5-build] glibc: None, default [dist-el5-build] glibc-common: None, default [dist-el5-build] glibc-devel: None, default [dist-el5-build] glibc-headers: None, default [dist-el5-build] gzip: None, default [dist-el5-build] info: None, default [dist-el5-build] libselinux: None, default [dist-el5-build] libsemanage: None, default [dist-el5-build] libsepol: None, default [dist-el5-build] libtool-ltdl: None, default [dist-el5-build] make: None, default [dist-el5-build] patch: None, default [dist-el5-build] perl: None, default [dist-el5-build] policycoreutils: None, default [dist-el5-build] python: None, default [dist-el5-build] readline: None, default [dist-el5-build] readline-devel: None, default [dist-el5-build] redhat-release: None, default [dist-el5-build] redhat-rpm-config: None, default [dist-el5-build] rpm-build: None, default [dist-el5-build] rpm-libs: None, default [dist-el5-build] sed: None, default [dist-el5-build] selinux-policy: None, default [dist-el5-build] shadow-utils: None, default [dist-el5-build] sqlite: None, default [dist-el5-build] tar: None, default [dist-el5-build] unzip: None, default [dist-el5-build] which: None, default [dist-el5-build] zip: None, default [dist-el5-build] zlib-devel: None, default [dist-el5-build] Wow, that's way more in your srpm-build group than necessary. srpm building is a fairly simple process, and doesn't require a lot in the buildroot. For reference, this is the Fedora srpm-build group: $ koji list-groups dist-f12-build [snip] srpm-build [dist-f9] bash: None, default [dist-f9-build] curl: None, default [dist-f9-build] cvs: None, default [dist-f9-build] fedora-release: None, default [dist-f9-build] gnupg: None, default [dist-f9-build] make: None, default [dist-f9-build] redhat-rpm-config: None, default [dist-f9-build] rpm-build: None, default [dist-f9-build] shadow-utils: None, default [dist-f9-build] Of course, any dependencies of those packages will also be installed, but using a smaller base package set will make your buildSRPMFromSCM tasks complete significantly faster. It appears that the repos have regenned and now when I try to do a build, I'm getting DEBUG util.py:280: Executing command: ['make', 'sources'] DEBUG util.py:256: make: *** No rule to make target `sources'. Stop. DEBUG util.py:319: Child returncode was: 2 Has a new make command "sources" been created similar to "make srpm"? If so what does it do and what does koji expect back? Brian -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Make sources?
Brian Kosick wrote: OK I added NOOP= true source: $(NOOP) To my make file and koji builds fine... Can I ask, what is the conceptual reason to have the source target? In Fedora CVS the upstream source tarball(s) are stored in a lookaside cache and only referenced from the CVS checkout. "make sources" pulls the tarball(s) into the checkout so "mock --buildsrpm" can use them to build the srpm without knowing the details of the source control layout. In your case, its fine to define the "sources" target as a noop. Calling "make sources" should probably be configurable at the tag level, but that feature hasn't been a priority. Brian On Fri, 2009-05-01 at 19:07 -0600, Brian Kosick wrote: Hi All, So I think that I have successfully upgraded koji from 1.2.6 to 1.3.1 by doing this. I just have a few issues to work out 1) install the koji 1.3.1 rpms 2) update the db schema pgsql -h kojihost koji koji < /usr/share/doc/koji-1.3.1/docs/schema-upgrade-1.2-1.3.sql 3) started kojid and kojira When I did my first build, I got stuck with koji not having a mock group srpm-build I setup the mock group with koji add-group dist-el5-build srpm-build koji add-group-pkg dist-el5-build srpm-build pkg1 pkg2 pk3 I then added that group to my build dist with koji call addGroupList dist-mxl-el5-build srpm-build When I do a koji list-groups dist-mxl-el5-build i get koji list-groups dist-mxl-el5-build build [dist-mxl-el5-build] srpm-build [dist-el5-build] automake: None, default [dist-el5-build] bash: None, default [dist-el5-build] buildsys-macros: None, default [dist-el5-build] bzip2: None, default [dist-el5-build] bzip2-devel: None, default [dist-el5-build] coreutils: None, default [dist-el5-build] cpio: None, default [dist-el5-build] diffutils: None, default [dist-el5-build] elfutils: None, default [dist-el5-build] elfutils-libelf: None, default [dist-el5-build] file: None, default [dist-el5-build] gcc: None, default [dist-el5-build] gcc-c++: None, default [dist-el5-build] glibc: None, default [dist-el5-build] glibc-common: None, default [dist-el5-build] glibc-devel: None, default [dist-el5-build] glibc-headers: None, default [dist-el5-build] gzip: None, default [dist-el5-build] info: None, default [dist-el5-build] libselinux: None, default [dist-el5-build] libsemanage: None, default [dist-el5-build] libsepol: None, default [dist-el5-build] libtool-ltdl: None, default [dist-el5-build] make: None, default [dist-el5-build] patch: None, default [dist-el5-build] perl: None, default [dist-el5-build] policycoreutils: None, default [dist-el5-build] python: None, default [dist-el5-build] readline: None, default [dist-el5-build] readline-devel: None, default [dist-el5-build] redhat-release: None, default [dist-el5-build] redhat-rpm-config: None, default [dist-el5-build] rpm-build: None, default [dist-el5-build] rpm-libs: None, default [dist-el5-build] sed: None, default [dist-el5-build] selinux-policy: None, default [dist-el5-build] shadow-utils: None, default [dist-el5-build] sqlite: None, default [dist-el5-build] tar: None, default [dist-el5-build] unzip: None, default [dist-el5-build] which: None, default [dist-el5-build] zip: None, default [dist-el5-build] zlib-devel: None, default [dist-el5-build] It appears that the repos have regenned and now when I try to do a build, I'm getting DEBUG util.py:280: Executing command: ['make', 'sources'] DEBUG util.py:256: make: *** No rule to make target `sources'. Stop. DEBUG util.py:319: Child returncode was: 2 Has a new make command "sources" been created similar to "make srpm"? If so what does it do and what does koji expect back? Brian -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mergerepos of f10 x86_64 release and updates does not contain perl or some other packages?
Steve Traylen wrote: Hi, Merging works fine for i386 f10 repositories but for x86_64 the following happens. The merged repository of f10's "Everything" and "updates" is created and then using this new repo mock tries to create and installroot. The problem is that this new repository is unable to provide some items, perl, /bin/bash and so the install root fails. Here are the details. The "blocklist" file following here is empty. $ /usr/libexec/kojid/mergerepos -a x86_64 \ -b /mnt/koji/repos/dist-f10-build/671/x86_64/blocklist \ -o /tmp/koji/tasks/2021/2021/repo \ -g /mnt/koji/repos/dist-f10-build/671/groups/comps.xml \ -r http://mirror.switch.ch/ftp/mirror/fedora/linux/updates/10/x86_64/ \ -r http://mirror.switch.ch/ftp/mirror/fedora/linux/releases/10/Everything/x86_64/os/ Adding repo: http://mirror.switch.ch/ftp/mirror/fedora/linux/updates/10/x86_64/ Adding repo: http://mirror.switch.ch/ftp/mirror/fedora/linux/releases/10/Everything/x86_64/os/ 1/12364 - openswan-doc-2.6.19-1.fc10.x86_64 2/12364 - ssm-0.1-12.fc10.x86_64 ... ... 1112/12364 - 4:perl-5.10.0-56.fc10.x86_64 ... ... 12363/12364 - libextractor-plugins-exiv2-0.5.20b-2.fc10.x86_64 12364/12364 - lwp-devel-2.4-1.fc10.x86_64 Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Starting other db creation: Tue Mar 17 14:49:44 2009 Ending other db creation: Tue Mar 17 14:50:50 2009 Starting filelists db creation: Tue Mar 17 14:51:03 2009 Ending filelists db creation: Tue Mar 17 14:55:37 2009 Starting primary db creation: Tue Mar 17 14:55:40 2009 Ending primary db creation: Tue Mar 17 14:58:28 2009 Sqlite DBs complete The following is then tried from within mock. The chroot's yum.conf is definitely referencing the repository I just created (I've checked carefully and have recreated the repo a few times just to check this is consistent. ) /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root/ \ groupinstall build this results in EBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:--> Missing Dependency: /usr/bin/perl is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:--> Missing Dependency: perl(Getopt::Long) is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:--> Missing Dependency: /bin/sh is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:--> Missing Dependency: mktemp is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: fedora-release-notes-10.0.0-1.noarch from build has depsolving problems DEBUG util.py:256:--> Missing Dependency: /bin/sh is needed by package fedora-release-notes-10.0.0-1.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:--> Missing Dependency: /bin/bash is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/bash is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: mktemp is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/sh is needed by package fedora-release-notes-10.0.0-1.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /usr/bin/perl is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/sh is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: perl(Getopt::Long) is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) And indeed. /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root list perl Error: No matching Packages to list There are lots of packages in this repository, but no perl it seems. /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root list | wc -l shows 3483 packages. Note the main box I am working on is 32 bit F10 box which may be relevant and the reason why only subsequently the i386 build works? You can't use a x86_64 chroot on a i386 box, nothing in the chroot will run. I suspect rpm is preventing installation of x86_64 packages on a i386 host. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: buildSRPMfromSCM issue
Jitesh Shah wrote: Hi all, I've been trying out some stuff on my own local koji instance. Recently, I added a new builder and all the builds using SCMs started to fail (although, scratch builds on SRPMs work just fine). buildSRPMfromSCM fails. The checkouts go through fine and "make sources" also works. Following is the tail of the root.log (after "make sources") DEBUG util.py:251: 3 2266k 73 1670k0 0 239k 0 0:00:09 0:00:06 0:00:03 310k 87 2266k 87 1976k0 0 247k 0 0:00:09 0:00:07 0:00:02 310k 100 2266k 100 2266k0 DEBUG util.py:251: DEBUG util.py:251: 0 DEBUG util.py:251: DEBUG util.py:251: DEBUG util.py:251: DEBUG util.py:251: 253k 0 0:00:08 0:00:08 --:--:-- 307k DEBUG util.py:251: -rw-rw-r-- 1 mockbuild mockbuild 2321402 Jun 18 2007 libidn-0.6.14.tar.gz DEBUG util.py:312: Child returncode was: 0 DEBUG backend.py:484: umount -n /var/lib/mock/dist-f10-build-139-697/root/proc DEBUG util.py:273: Executing command: umount -n /var/lib/mock/dist-f10-build-139-697/root/proc DEBUG util.py:312: Child returncode was: 0 DEBUG backend.py:484: umount -n /var/lib/mock/dist-f10-build-139-697/root/sys DEBUG util.py:273: Executing command: umount -n /var/lib/mock/dist-f10-build-139-697/root/sys DEBUG util.py:312: Child returncode was: 0 DEBUG util.py:99: kill orphans The task exits with "FAILED: BuildError: error building srpm, mock exited with status 2; see root.log for more information" But, there is nothing suspicious in root.log (yum succeeds, so does checkout and "make sources" and there are no errors after that too). state.log and kojid.log also give out nothing. I have enabled full debugging from kojihub.conf. That error means "mock --buildsrpm" failed. I'm not sure what status 2 is, it probably means a failure in rpmbuild in the chroot. There should be more information in root.log, I'm not sure why there isn't. Does building the srpm by hand work? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: how to build srpms on koji?
陈鲍孜 wrote: I checked the /var/log/kojid.log. it is said: 2009-03-14 17:03:14,371 [WARNING] koji.build.TaskManager: TRACEBACK: Traceback (most recent call last): File "/usr/sbin/kojid", line 1275, in runTask response = (handler.run(),) File "/usr/sbin/kojid", line 1351, in run return self.handler(*self.params,**self.opts) File "/usr/sbin/kojid", line 2560, in handler for f in os.listdir(self.datadir): OSError: [Errno 2] No such file or directory: '/tmp/koji/tasks/230/230/repo/repoda 2009/3/15 Steve Traylen If your build tag has no builds associated or inherited, and no external repos configured, then no repodata will be created and the task will fail this way when it tries to upload the repodata. Tag a package into the build tag, or enable an external repo. On Sun, Mar 15, 2009 at 10:55 AM, 陈鲍孜 wrote: Hi, After long time fighting with koji configuration, I think the service is now running successfully. I added some srpms to it and saw its information on kojiweb. But it seems I can not build the srpms by koji (when I submit a build task to koji, it will "open" for a while, then it would finally come to be failed). I doubted if I have missed some configurations which connecting koji and mock. Does the build get as far as kojid. Is there anything in /var/log/kojid.log ? This can tell you the mock problem and then where to look for that. Steve -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mergerepo fails with PCDATA invalid Char value 8
Steve Traylen wrote: On Sun, Mar 15, 2009 at 12:45 PM, Steve Traylen wrote: Hi, Got koji basically working for me over the last couple of weeks. Was very keen to try its new external repository support. Starting with a fresh instance I made a tag (dist-slc5) containing two repos. koji add-external-repo -t dist-slc5 -p 10 "slc5-64-base" http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os koji add-external-repo -t dist-slc5 -p 10 "slc5-32-base" http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os and then tried to make a koji repo from that. koji regen-repo dist-slc5 This called /usr/libexec/kojid/mergerepos -a i386 -b /mnt/koji/repos/dist-slc5-build/189/i386/blocklist -o /tmp/koji/tasks/556/556/repo \ -g /mnt/koji/repos/dist-slc5-build/189/groups/comps.xml -r http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os/ \ -r http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os/ resulting in as below. Any ideas ? To hopefully answer my own question. Is this because these slc yum repositories do not contain the sqlite files thats that mergerepo makes use of. Looking at CentOS and ScientificLinux neither of these look to make use of the '-d' option to createrepo to generate the sqlite files. Is there a way around this or we have to ask CentOS to generate the sql files? The error occurs when parsing other.xml. I would check your external repos to see if other.xml passes XML validation successfully. Of course maybe it is something else entirely? Steve Steve process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 Traceback (most recent call last): File "/usr/libexec/kojid/mergerepos", line 241, in main(sys.argv[1:]) File "/usr/libexec/kojid/mergerepos", line 236, in main merge.write_metadata() File "/usr/libexec/kojid/mergerepos", line 216, in write_metadata mdgen.doPkgMetadata() File "/usr/lib/python2.5/site-packages/createrepo/__init__.py", line 332, in doPkgMetadata self.writeMetadataDocs(packages) File "/usr/lib/python2.5/site-packages/createrepo/__init__.py", line 475, in writeMetadataDocs clog_limit=self.conf.changelog_limit)) File "/usr/lib/python2.5/site-packages/yum/packages.py", line 959, in xml_dump_other_metadata msg += "%s\n\n" % misc.to_unicode(self._dump_changelog(clog_limit)) File "/usr/lib/python2.5/site-packages/yum/packages.py", line 927, in _dump_changelog if not self.changelog: File "/usr/lib/python2.5/site-packages/yum/packages.py", line 423, in changelog = property(fget=lambda self: self.returnChangelog()) File "/usr/lib/python2.5/site-packages/yum/sqlitesack.py", line 225, in returnChangelog self._loadChangelog() File "/usr/lib/python2.5/site-packages/yum/sqlitesack.py", line 202, in _loadChangelog self.sack.populate(self.repo, mdtype='otherdata') File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 184, in populate dobj = repo_cache_function(xml, csum) File "/usr/lib/python2.5/site-packages/sqlitecachec.py", line 60, in getOtherdata self.repoid)) TypeError: Parsing other.xml error: PCDATA invalid Char value 8 Steve -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: koji 1.3.0
Brian Schubert wrote: > Hi, > > Is there an easy way to convert the database from Koji 1.2.6 to work > with this new release? > > Importing the initial packages took a very long time, and as such, > starting with a fresh database is undesirable. Or has the import speed > improved drastically with this release? There is an upgrade script (/usr/share/doc/koji-1.3.1/docs/schema-upgrade-1.2-1.3.sql) to upgrade an existing database the 1.3 format. Note that the external repos support may allow you to avoid an initial import altogether, in many cases. There is documentation forthcoming on how to setup external repos. In the meantime you can ask questions in #koji on Freenode. > Thanks, > > -Brian Schubert > > Mike McLean wrote: >> The tarball is posted here: >> https://fedorahosted.org/koji/wiki/KojiRelease >> (created from koji-1.3.0 tag) >> >> >> Major Changes >> -- >> Support for external repos >> new cli commands >> createrepo tasks now have a higher weight >> Support for noarch subpackages >> Support larger hashes >> Handle different kinds of rpm signatures [mitr] (#127) >> support file digests other than md5 in the api and web UI >> Hub configuration via hub.conf >> use a hub config file instead of PythonOptions >> scan for handlers once at startup instead of each call >> now honors KojiDir >> Remove huge tables from db >> - rpmfiles, rpmdeps and changelog tables dropped >> - data pulled from rpms as needed >> Build srpms in chroots >> bump the weight of buildSRPMFromSCM >> Some web ui theming support >> SiteName option (the name that shows up in the title) >> move explicit image sizes to css >> make welcome message customizable >> Improved security >> add an auth token to defend against CSRF >> include the current time in the cookie >> some changes to the web UI to defend against XSS >> Hub policies >> configured in hub.conf >> enable verbose policy errors without full debug >> current policies: >> tag: controls tag, untag, move operations >> build_from_srpm: controls whether such builds are allowed >> build_from_repo_id: controls whether such builds are allowed >> Plugin support >> kojihub and kojid now have limited plugin support >> >> >> Other Changes >> - >> Python-2.6-isms >> use hashlib if available >> use subprocess instead of popen2 >> Builder >> improved task cleanup >> check the load average before taking a task >> make use of createrepo --update optional >> choose a better arch for noarch builds >> update the buildArch task weight based on the average duration of >> a build >> of the package >> set %distribution the same way we set %vendor and %packager >> cleanup before re-taking a freed or reassigned task >> indicate which log people should look in for build failures >> make use of --skip-stat configurable >> use same repo for all buildArch subtasks >> more fully honor topdir option >> Web UI >> show summary and description for rpms and builds >> group rpms more sensibly (make the build log links correct) >> remove the dirtyness indicator from the buildrootinfo page (never >> used) >> enable displaying of only the latest builds in a tag >> use ids instead of names in the urls >> fix the "Watch logs" feature in the web UI to work over SSL >> cache compiled Cheetah templates >> update the web UI to conform to XHTML 1.0 Transitional >> tasks page: rework view selector >> use kojiweb.publisher (new location) >> Hub >> NotifyOnSuccess option >> honor KojiDir option >> DisableNotifications option >> don't blow up when the database contains older base64 encoded task >> data >> make a latest link when a new repo completes >> add createdBefore= and createdAfter= parameters to listBuilds() >> fix LoginCreatesUser check >> in resetBuild use CANCELED state instead of FAILED >> report offline status if: >> db connection fails >> there are errors on startup >> preserve old extra_arches during package list updates >> Command line >> new subcommand: search >> new subcommand: regen-repo >> new subcommand: remove-tag >> show package filters correctly in taginfo >> allow list-tagged to query at an event >> added --non-latest to untag-pkg subcommand >> added --old-user flag to set-pkg-owner-global subcommand >> show buildroot info in rpminfo output >> show arch in list-buildroot output >> handle chain-build cases where the build tag is the same as the >> dest tag >> Utils >> don't start kojira by default (#96) >> fix timestamp checks when deleting repos >> package koji-gc >> added purge option to koji-gc >> added koji-shadow utility for shadow builds >> Changes related to shadow builds >> koji-shadow utility >> allow creation of repos from a specified event >> all
Re: kojira repo generation
Thomas Hatch wrote: > I keep having problems with it telling me the system is locked until I run a > restart, but service kojid status keeps returning the same error > > service kojid status > kojid dead but subsys locked > > kojid also seems to be dying but the logs yield no real data > > I think I have a problem in my configs: What is the output of openssl x509 -noout -subject -in /etc/pki/koji/kojibuilder1.pem The CN component needs to match the hostname you added with "koji add-host", in your case koji.bcinfra.net. Also, that same certificate may not be used to authenticate any other services or users to the system. You can also run /usr/sbin/kojid --force-lock --verbose --fg as root to run kojid in the foreground and see what errors are reported. > kojid.conf: > > [kojid] > ; The number of seconds to sleep between tasks > ; sleeptime=15 > > ; The maximum number of jobs that kojid will handle at a time > ; maxjobs=10 > > ; The minimum amount of free space (in MBs) required for each build root > ; minspace=8192 > > ; The directory root where work data can be found from the koji hub > ; topdir=/mnt/koji > > ; The directory root for temporary storage > workdir=/tmp/koji > > ; The directory root for mock > mockdir=/var/lib/mock > > ; The user to run as when doing builds > mockuser=kojibuilder > > ; The vendor to use in rpm headers > ; vendor=Koji > > ; The packager to use in rpm headers > ; packager=Koji > > ; The _host string to use in mock > ; mockhost=koji-linux-gnu > > ; The URL for the xmlrpc server > server=http://sunlight.pp.bcinfra.net/kojihub > > user=koji.bcinfra.net > > ; The URL for the packages tree > pkgurl=http://sunlight.pp.bcinfra.net/pkg/packages > > ; A space-separated list of hostname:repository[:use_common] tuples that > kojid is authorized to checkout from (no quotes). > ; Wildcards (as supported by fnmatch) are allowed. > ; If use_common is specified and is one of "false", "no", or "0" (without > quotes), then kojid will not attempt to checkout > ; a common/ dir when checking out sources from the source control system. > Otherwise, it will attempt to checkout a common/ > ; dir, and will raise an exception if it cannot. > ;allowed_scms=scm.example.com:/cvs/example git.example.org:/example > svn.example.org:/users/*:no > > ; The mail host to use for sending email notifications > smtphost=sunlight.pp.bcinfra.net > > ; The From address used when sending email notifications > from_addr=Koji Build System > > ;configuration for SSL athentication > > ;client certificate > cert = /etc/pki/koji/kojibuilder1.pem > > ;certificate of the CA that issued the client certificate > ca = /etc/pki/koji/koji_ca_cert.crt > > ;certificate of the CA that issued the HTTP server certificate > serverca = /etc/pki/koji/koji_ca_cert.crt > > > > > > > On Thu, Feb 26, 2009 at 10:32 AM, Jeffrey Ollie wrote: > >> On Thu, Feb 26, 2009 at 11:29 AM, Thomas Hatch wrote: >>> I run "koji list-hosts --channel=createrepo" and get: >>> >>> Hostname Enb Rdy Load/Cap Arches Last >> Update >>> koji.bcinfra.net Y N0.0/8.0 i386,x86_64 - >>> >>> Seems it is enabled and in the channel, but not ready? >> Is kojid running? That's the service that does the actual building... >> >> -- >> Jeff Ollie >> Marcus to Franklin in Babylon 5: "A Late Delivery from Avalon" >> >> -- >> Fedora-buildsys-list mailing list >> Fedora-buildsys-list@redhat.com >> https://www.redhat.com/mailman/listinfo/fedora-buildsys-list >> > > > > > -- > Fedora-buildsys-list mailing list > Fedora-buildsys-list@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: attribute errors when koji login
陈鲍孜 wrote: > Hello, > > After fighting with my carelessness, I finished configuring koji and made it > run. Thanks a lot. > > Now, here comes another problem. When I click the "login link" on the > kojiweb, it returns an error such as below: > > Traceback (most recent call last): > > File "/usr/lib64/python2.4/site-packages/mod_python/apache.py", line 299, > in HandlerDispatch > result = object(req) > > File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py", line > 213, in handler > published = publish_object(req, object) > > File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py", line > 412, in publish_object > return publish_object(req,util.apply_fs_data(object, req.form, req=req)) > > File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line 439, in > apply_fs_data > return object(**args) > > File "/usr/share/koji-web/scripts/index.py", line 155, in login > _redirectBack(req, dest, forceSSL=True) > > File "/usr/share/koji-web/scripts/index.py", line 134, in _redirectBack > page = _getBaseURL(req) + '/' + page > > File "/usr/share/koji-web/scripts/index.py", line 117, in _getBaseURL > return req.construct_url(base) > > AttributeError: 'mp_request' object has no attribute 'construct_url' > > I searched google. There are some results, but still not clear for me. Is it > a bug or something due to some wrong configuration? I was run koji on CentOS > with EPEL packages. What version of Koji are you using? That error has been fixed in Koji git for a while, and is available in the latest release, Koji 1.3.1. You'll need to run it on CentOS 5. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] Add compatibility back in for older versions of Yum.
James Antill wrote:
> On Tue, 2009-02-17 at 10:29 -0600, Jeffrey C. Ollie wrote:
>> We would like for Koji to remain compatible with the version of Yum
>> that is shipped with RHEL 5.
>>
>> Signed-off-by: Jeffrey C. Ollie
>> ---
>> builder/mergerepos | 11 +++
>> 1 files changed, 7 insertions(+), 4 deletions(-)
>>
>> diff --git a/builder/mergerepos b/builder/mergerepos
>> index defb8c1..cfa3a8d 100755
>> --- a/builder/mergerepos
>> +++ b/builder/mergerepos
>> @@ -95,10 +95,13 @@ class RepoMerge(object):
>> self.mdconf.verbose = True
>> self.mdconf.changelog_limit = 3
>> self.yumbase = yum.YumBase()
>> -self.yumbase.preconf.fn = '/dev/null'
>> -self.yumbase.preconf.init_plugins = False
>> -self.yumbase.preconf.debuglevel = 2
>> -self.yumbase._getConfig()
>> +if hasattr(self.yumbase, 'preconf'):
>> +self.yumbase.preconf.fn = '/dev/null'
>> +self.yumbase.preconf.init_plugins = False
>> +self.yumbase.preconf.debuglevel = 2
>> +self.yumbase._getConfig()
>
> This line should be deleted, the first line after the if will turn the
> preconf into the conf.
Thanks, I've made that change.
>> +else:
>> +self.yumbase._getConfig('/dev/null', init_plugins=False,
>> debuglevel=2)
>> self.yumbase.conf.cachedir = tempfile.mkdtemp()
>> self.yumbase.conf.cache = 0
>> self.archlist = arches
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] Add compatibility back in for older versions of Yum.
Jeffrey C. Ollie wrote:
We would like for Koji to remain compatible with the version of Yum
that is shipped with RHEL 5.
Signed-off-by: Jeffrey C. Ollie
---
builder/mergerepos | 11 +++
1 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/builder/mergerepos b/builder/mergerepos
index defb8c1..cfa3a8d 100755
--- a/builder/mergerepos
+++ b/builder/mergerepos
@@ -95,10 +95,13 @@ class RepoMerge(object):
self.mdconf.verbose = True
self.mdconf.changelog_limit = 3
self.yumbase = yum.YumBase()
-self.yumbase.preconf.fn = '/dev/null'
-self.yumbase.preconf.init_plugins = False
-self.yumbase.preconf.debuglevel = 2
-self.yumbase._getConfig()
+if hasattr(self.yumbase, 'preconf'):
+self.yumbase.preconf.fn = '/dev/null'
+self.yumbase.preconf.init_plugins = False
+self.yumbase.preconf.debuglevel = 2
+self.yumbase._getConfig()
+else:
+self.yumbase._getConfig('/dev/null', init_plugins=False,
debuglevel=2)
self.yumbase.conf.cachedir = tempfile.mkdtemp()
self.yumbase.conf.cache = 0
self.archlist = arches
Applied, thanks for the patch!
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
[PATCH] mock: set the HOME environment variable
Signed-off-by: Mike Bonnet
---
Set the HOME environment variable to match the chroothome config
parameter. Allows commands running under --chroot --unpriv to create
dotfiles, etc.
py/mock.py |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/py/mock.py b/py/mock.py
index d04c859..677691b 100755
--- a/py/mock.py
+++ b/py/mock.py
@@ -410,7 +410,6 @@ def main(ret):
uidManager = mock.uid.uidManager(unprivUid, unprivGid)
uidManager._becomeUser(unprivUid, unprivGid)
-del(os.environ["HOME"])
# defaults
config_opts = {}
@@ -498,6 +497,7 @@ def main(ret):
ret["chroot"] = chroot
ret["config_opts"] = config_opts
os.umask(002)
+os.environ["HOME"] = chroot.homedir
# New namespace starting from here
try:
--
1.6.0.6
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji feature proposals
On Tue, 2009-01-06 at 15:21 -0600, Jason L Tibbitts III wrote: > > "OF" == Oliver Falk writes: > > OF> And regarding your point: '... different arches build noarch > OF> subpackage with different contents'. Well, then it's definitly not > OF> *noarch*, is't it? :-) > > It is quite possible for the contents to differ by, say, date, or by > timestamps being included in plain text output. Why would that render > the output arch-specific? I'm not so much worried about that level of difference as I am of say different file lists from noarch rpms built on different hosts, or maybe different endianness of data files. There is some set of post-build checks we may want to run on these noarch subpackages to ensure they are in fact noarch, and that their content is sane. This noarch subpackage feature is new, and there are things Koji can and probably should be doing to make sure it's being used correctly. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji feature proposals
On Tue, 2009-01-06 at 21:51 +0100, Oliver Falk wrote: > Hi Mike! > > Mike Bonnet schrieb: > > I've just created tickets for a few Koji features that I've been wanting > > to implement for a while (as well as updated an old one), and I'm > > planning to devote some time to in the near future. If you have any > > comments on these features feel free to post to the tickets, or talk to > > me at FUDCon this weekend. Just figured people might want to see the > > direction that Koji is headed. The future is now! :) > > [ ... ] > > > drop the rpmfiles and rpmdeps tables: > > https://fedorahosted.org/koji/ticket/124 > > -1 > Only if you provide the same functionality using another approach :-) Yes, the plan is to query the information directly from the rpms rather than from the database. The content on the rpminfo page in the web UI should not change at all from the user perspective. > *I* do use it quite often; Find out which file belongs to which pkg(s). > > However, this was quite slow and now doesn't even seem to work in > koji.fpo. :-( Hmmm, it should be. In what way is it not working? > > noarch subpackage support: https://fedorahosted.org/koji/ticket/125 > > Duh? We do have already (at least one) packages that build arch-specific > and noarch pkgs -> kernel or do we use some *hack* in the kernel.spec? We use a hack in the kernel specfile and in the build system. The noarch subpackage support in rpm is much more generic and flexible, and we need to support it without build system hacks. > And regarding your point: '... different arches build noarch subpackage > with different contents'. Well, then it's definitly not *noarch*, is't > it? :-) True, but it's still possible, and we may need to check for this case and handle is appropriately (possibly by failing the build). -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Koji feature proposals
I've just created tickets for a few Koji features that I've been wanting to implement for a while (as well as updated an old one), and I'm planning to devote some time to in the near future. If you have any comments on these features feel free to post to the tickets, or talk to me at FUDCon this weekend. Just figured people might want to see the direction that Koji is headed. The future is now! :) Thanks, Mike build the srpm in a chroot: https://fedorahosted.org/koji/ticket/103 support for building isos and disk images in Koji: https://fedorahosted.org/koji/ticket/122 koji callback system: https://fedorahosted.org/koji/ticket/123 drop the rpmfiles and rpmdeps tables: https://fedorahosted.org/koji/ticket/124 noarch subpackage support: https://fedorahosted.org/koji/ticket/125 -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Supporting EPEL Builds in Koji
Picking up this thread again, sorry about the long delay. I'd like to come to consensus on the approach here, hammer out any remaining details at FUDCon this weekend, and hopefully get this implemented by the end of January. Time to really get rid of plague! On Mon, 2008-10-06 at 15:14 -0400, Mike McLean wrote: > Mike Bonnet wrote: > > On Fri, 2008-07-18 at 11:38 -0400, Mike McLean wrote: > >> Mike Bonnet wrote: > >>> On Thu, 2008-07-17 at 13:54 -0400, Mike McLean wrote: > >>>> If the remote_repo_url data is going to be inherited (and I tend to > >>>> think it should be), then I think it should be in a separate table. > ... > >>> I don't have any problem with this, though it does mean we'll need to > >>> duplicate quite a bit of the inheritance-walking code, > ... > >> Walking inheritance is just a matter of determining the inheritance > >> order and scanning data on the parent tags in sequence. > ... > > Sorry, I was referring to walking tag_inheritance. I'd rather have one > > place that walks the inheritance hierarchy and aggregates data from it, > > than two places that are doing almost the same thing. > > We're talking about inherently different data. External repos to be > merged in are quite different from builds in the system. Yes, I see the issue here. Since remote repos won't have their packages filtered out (by mergerepo) until after all packages in the local inheritance hierarchy are placed in the repo, they don't really follow the existing inheritance rules. Ok, you've convinced me. A separate table that stores a priority-ordered list of remote repos associated with each tag will probably be easier to manage. The lists will be aggregated when walking the tag hierarchy and passed to mergerepo in (priority, inheritance) order for proper filtering (based on srpm name, first match wins). > > Each tag has a set of builds associated with it. We walk the > > inheritance hierarchy, aggregating the builds from each tag in the > > hierarchy into a flat list, and then pass that list to createrepo. We > > would do essentially the same thing for external repos. When walking > > the hierarchy, if a tag has an external repo associated with it, we > > would append that repo url to a flat list, and pass that list to > > mergerepo. In both cases we're working with collections of packages > > that are associated with a tag, just in different formats. > > Sure, we can do this with one call to readFullInheritance, and traverse > both the build table and external repo table from the given order. Yes, that makes sense. > > In discussing this with Jesse, I think we want external repos to be > > inherited. This is probably the easiest way to deal with having > > multiple external repos getting pulled in to a single buildroot, which > > is essential for Fedora (think F9 GA and F9 Updates). > > > > The idea was that, by convention, we would have external-repo-only tags, > > with only a single external repo associated with it and no > > packages/builds associated. These external-repo-only tags could then be > > inserted into the build hierarchy where appropriate. An ordered list of > > external repos could then be constructed by performing the current > > depth-first search of the inheritance hierarchy. The ordered list would > > then be passed to mergerepo, which would ensure that packages in repos > > earlier in the list supersede packages (by srpm name) in repos later in > > the list. This would preserve the "first-match-wins" inheritance policy > > that Koji currently implements, and that admins expect. For example: > > > > dist-custom-build > > ├─dist-custom > > └─dist-f9-updates-external > > └─dist-f9-ga-external > > > > would result mergerepo creating a single repo that would only contain > > packages from dist-f9-ga-external if they did not exist in the > > Koji-generated repo (dist-custom-build + dist-custom), > > dist-f9-updates-external, or the blacklist of blocked packages. This is > > consistent with how Koji package inheritance currently works, and I > > think is the most intuitive approach. > > It is similar, but different in potentially confusing ways. External > repos do not have build structure, so we can't really have the same sort > of inheritance behavior with a combination of external repo tags and > normal tags. > > We order the external repos in inheritance order, but ultimately those > repos are merged with the internal one in a way that does not honor > inheritance in the way that the admin might expect. >
Re: caching packages on koji builder
On Wed, 2008-11-05 at 09:49 -0600, Jason L Tibbitts III wrote: > >>>>> "MB" == Mike Bonnet <[EMAIL PROTECTED]> writes: > > MB> Koji builders have never downloaded packages via XMLRPC. All > MB> downloading is done by mock/yum, via http (previously nfs). > > Well, mock can cache all sorts of things these days. If there are > multiple builders at one location then having a single squid cache for > them all might be nice, but mock's caching would still help to avoid > having to hit the network. Actually mock's caching doesn't really help us. It's all done per-buildroot, and since every build is run in a different buildroot, the caches would never be reused. For this reason Koji disables caching in the mock configs it writes out. A global (per-machine) rpm cache might be useful for reducing network bandwidth. However, because mock/yum would have to lock this global cache while interacting with it, it would become a bottleneck when running concurrent builds. The best approach currently is probably a local squid cache. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: caching packages on koji builder
On Wed, 2008-11-05 at 10:55 +0100, Dan Horák wrote: > Hello, > > is it possible to somehow cache the packages that are going from koji > hub to the builder to create a build root? There is no NFS connection > between the builder and hub in s390's koji and so each build requires to > download 100 MB via XMLRPC(?) and this is slow. Will squid help here? Koji builders have never downloaded packages via XMLRPC. All downloading is done by mock/yum, via http (previously nfs). You could potentially use squid locally to cache downloaded packages. You'd configure pkgurl in koji.conf to point to your local squid instance at "http://localhost:8080/koji/packages"; or something similar, and configure squid to pull from the actual http location where /mnt/koji/packages is being served. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Supporting EPEL Builds in Koji
On Fri, 2008-07-18 at 11:38 -0400, Mike McLean wrote: > Mike Bonnet wrote: > > On Thu, 2008-07-17 at 13:54 -0400, Mike McLean wrote: > >> If the remote_repo_url data is going to be inherited (and I tend to > >> think it should be), then I think it should be in a separate table. I'd > >> like to reserve tag_config for data that is local to individual tags. > >> This will also make it easier to represent multiple remote repos. > > > > I don't have any problem with this, though it does mean we'll need to > > duplicate quite a bit of the inheritance-walking code, or make it > > configurable as to which inheritance it's walking. This new table would > > also have to be versioned, the same way the tag_config table is. > > Walking inheritance is just a matter of determining the inheritance > order and scanning data on the parent tags in sequence. Currently, > nothing scans tag_config in this way because no data in tag_config is > inherited. (Well, in a sense tag_changed_since_event() does walk > tag_config, but that's a little different.) Sorry, I was referring to walking tag_inheritance. I'd rather have one place that walks the inheritance hierarchy and aggregates data from it, than two places that are doing almost the same thing. Each tag has a set of builds associated with it. We walk the inheritance hierarchy, aggregating the builds from each tag in the hierarchy into a flat list, and then pass that list to createrepo. We would do essentially the same thing for external repos. When walking the hierarchy, if a tag has an external repo associated with it, we would append that repo url to a flat list, and pass that list to mergerepo. In both cases we're working with collections of packages that are associated with a tag, just in different formats. > We need to figure out how we'll deal with multiplicity for the external > repos. If tag A uses repo X and inherits from tag B which uses repo Y, > then does tag A use both X and Y, or does the X entry override it? > A (+repo X) > +- B (+repo Y) > > My inclination is that it should override, because I think we'll want > some way to do override that that mechanism seems easiest. In discussing this with Jesse, I think we want external repos to be inherited. This is probably the easiest way to deal with having multiple external repos getting pulled in to a single buildroot, which is essential for Fedora (think F9 GA and F9 Updates). The idea was that, by convention, we would have external-repo-only tags, with only a single external repo associated with it and no packages/builds associated. These external-repo-only tags could then be inserted into the build hierarchy where appropriate. An ordered list of external repos could then be constructed by performing the current depth-first search of the inheritance hierarchy. The ordered list would then be passed to mergerepo, which would ensure that packages in repos earlier in the list supersede packages (by srpm name) in repos later in the list. This would preserve the "first-match-wins" inheritance policy that Koji currently implements, and that admins expect. For example: dist-custom-build ├─dist-custom └─dist-f9-updates-external └─dist-f9-ga-external would result mergerepo creating a single repo that would only contain packages from dist-f9-ga-external if they did not exist in the Koji-generated repo (dist-custom-build + dist-custom), dist-f9-updates-external, or the blacklist of blocked packages. This is consistent with how Koji package inheritance currently works, and I think is the most intuitive approach. > Also, I think we'll probably want to allow multiple external repos per > tag, something which will be much easier to represent in an external > table. We can include an explicit priority field to make a sane > uniqueness condition (and to provide a clear ordering for the repo merge). As outlined above, I'd prefer to keep it to one external repo per tag, along with repo inheritance. I think this is easier from a management perspective, and more consistent with the way Koji currently works. Ordering for mergerepo will be represented by the location of the tag in the inheritance hierarchy. With a 1-to-1 tag->external repo mapping, it then makes sense to store the external repo url in the tag_config table. > > The big win here is that the methods and tools that query rpminfo for > > information about what was present in the buildroot at build time > -snip- > > I see all that, and I'm almost convinced. The flipside is that by > default all the code will treat these external rpms the same as the > local ones, which will not be correct for a number of cases. Obviously, > part of this will involve changing code to behave differently for the &
Re: building from git using custom koji install
On Thu, 2008-07-24 at 09:32 -0400, Jesus Rodriguez wrote: > On Wed, Jul 23, 2008 at 11:51:36PM -0400, Mike Bonnet wrote: > > On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote: > > > On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote: > > > > On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote: > > > > > Jesus M. Rodriguez wrote: > > > > >> So am I correct in my assumption that koji expects the scm > > > > >> repository to house a single package? > > > > > > > > > > Yes. Furthermore, koji assumes that the simple command 'make srpm' > > > > > issued from the checkout will create a single source rpm. There is > > > > > not > > > > > enough information passed to koji for it to handle anything otherwise. > > > > > > > > > > In your SCM layout with multiple packages, how do you generate the > > > > > different srpms? Is there a real reason that they are all in one > > > > > module? > > > > > > > > We generate different srpms with the Makefile that exists in each > > > > subdirectory. For example, if you want an srpm for the web module > > > > cd spacewalk.git/web; make srpm > > > > > > > > Want one for the java code, then cd spacewalk.git/java. > > > > > > > > With SVN it seems completely doable because I could give the > > > > path to repo/spacewalk/java and have koji checkout just that > > > > directory. Seems to be a limitation of GIT in this case > > > > (or we're using it wrong) :) > > > > > > Well, git was intentionally written to be basically a state machine of > > > the entire project. So, housing more than one project in a single git > > > repo isn't wrong, but it does tie your various projects together at the > > > state machine level. This is why you can't checkout just one > > > subdirectory from git, you have to grab the entire project. I think I > > > heard mumblings of someone wanting to make subprojects work nicely in > > > git, but I haven't heard if it ever got off the ground. > > > > > > Regardless though, koji won't work with what your current SCM layout > > > without modification to the koji code. > > > > While that's true, I'm not sure we're definitely handling things > > correctly in the git case. The git support in Koji hasn't gotten a lot > > of testing, and there are a lot of ways to setup a git repo. However, I > > think that multiple projects in the same repo is probably something we > > want to support (since we support it in cvs and svn). > > > > The SCM URL you pass to Koji has a format of: > > > > scheme://[EMAIL > > PROTECTED]/path/to/repo?path/to/module#revision_or_tag_identifier > > > > With git we combine the /path/to/repo and the path/to/module into a > > single URL we pass to "git clone", essentially collapsing them both into > > a full module path (this is also what we do with svn). Would it be more > > appropriate to treat the /path/to/repo as the location of the git repo > > which we would pass to "git clone", and /path/to/module as a > > subdirectory in the repo, which we would use as the base directory of > > the build, and look there for a .spec file? > > Yep, that would actually work in our case. Because then I could do the > following: > > git://git.fedorahosted.org/git/spacewalk.git?web#HEAD > > web would contain the .spec file. I've just committed a change to Koji git to support this style of repo setup. The module path (everything between the ? and #) is now considered a subdirectory of the repo, and Koji will expect to find the relevant Makefile and specfile there. Building from the previous repo layout (where the top-level directory contains the Makefile and specfile) can be achieved by leaving empty or omitting the module path, like so: git://git.fedorahosted.org/git/koji?#a9a82be4 git://git.fedorahosted.org/git/koji#a9a82be4 The two examples above are functionally identical. I've tested these changes against both the Spacewalk and Koji git repos, and they seem to be working as expected. Let me know if you run into problems. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] make authtype a config option
On Mon, 2008-08-11 at 22:54 -0500, Dennis Gilmore wrote: > the attached patch adds a config option that can be in a config file or on > the > command line forcing the use of one authentication type. it is useful if a > hub supports more than one authentication type. or using different hubs that > support different authentications methods. Ive tested with noauth, kerberos, > and ssl. Applied. Thanks for the patch! -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: building from git using custom koji install
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote: > On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote: > > On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote: > > > Jesus M. Rodriguez wrote: > > >> So am I correct in my assumption that koji expects the scm > > >> repository to house a single package? > > > > > > Yes. Furthermore, koji assumes that the simple command 'make srpm' > > > issued from the checkout will create a single source rpm. There is not > > > enough information passed to koji for it to handle anything otherwise. > > > > > > In your SCM layout with multiple packages, how do you generate the > > > different srpms? Is there a real reason that they are all in one module? > > > > We generate different srpms with the Makefile that exists in each > > subdirectory. For example, if you want an srpm for the web module > > cd spacewalk.git/web; make srpm > > > > Want one for the java code, then cd spacewalk.git/java. > > > > With SVN it seems completely doable because I could give the > > path to repo/spacewalk/java and have koji checkout just that > > directory. Seems to be a limitation of GIT in this case > > (or we're using it wrong) :) > > Well, git was intentionally written to be basically a state machine of > the entire project. So, housing more than one project in a single git > repo isn't wrong, but it does tie your various projects together at the > state machine level. This is why you can't checkout just one > subdirectory from git, you have to grab the entire project. I think I > heard mumblings of someone wanting to make subprojects work nicely in > git, but I haven't heard if it ever got off the ground. > > Regardless though, koji won't work with what your current SCM layout > without modification to the koji code. While that's true, I'm not sure we're definitely handling things correctly in the git case. The git support in Koji hasn't gotten a lot of testing, and there are a lot of ways to setup a git repo. However, I think that multiple projects in the same repo is probably something we want to support (since we support it in cvs and svn). The SCM URL you pass to Koji has a format of: scheme://[EMAIL PROTECTED]/path/to/repo?path/to/module#revision_or_tag_identifier With git we combine the /path/to/repo and the path/to/module into a single URL we pass to "git clone", essentially collapsing them both into a full module path (this is also what we do with svn). Would it be more appropriate to treat the /path/to/repo as the location of the git repo which we would pass to "git clone", and /path/to/module as a subdirectory in the repo, which we would use as the base directory of the build, and look there for a .spec file? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: package upload/import speed
On Wed, 2008-07-23 at 14:03 -0700, Toshio Kuratomi wrote: > Mike Bonnet wrote: > > On Wed, 2008-07-23 at 15:56 -0400, Naveen Gavini wrote: > >> Hello all, > >> > >> When importing or building RPM's from source is there an option that is > >> throttling the upload speed of the package? > >> We are seeing speeds of 10kbps which makes importing/building large > >> packages very slow. > > > > Package upload happens via xml-rpc, and to avoid holding the entire > > contents of the file in memory, it's done in chunks. Unfortunately > > python xml-rpc libs don't support keep-alive, so if you're using SSL > > authentication you incur the non-trivial SSL connection setup/teardown > > overhead for every chunk, in addition to the TCP connection > > setup/teardown. The chunk size in your version is probably very small > > (64k?). That has been increased in the Koji repo, but not released yet. > > The next version will improve this situation somewhat. > > > Any chance we could port to pycurl which does support keep-alive? Or is > urllib pretty deeply embedded in the code ATM? I don't know anything about pycurl, but the transport layer is decently separated from the XML-RPC layer in xmlrpclib, so it should be possible to replace that it with a transport that supports SSL and keep-alive. I haven't looked at how much work this might be though. > -Toshio > > -- > Fedora-buildsys-list mailing list > Fedora-buildsys-list@redhat.com > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Problems using import --link
On Wed, 2008-07-23 at 16:33 -0400, Naveen Gavini wrote:
> Hello all,
>
> We are trying to bootstrap our new koji setup and when we attempt to
> import using --link we get the error below.
> The packages that we are attempting to import are on the hub and have
> been places in /mnt/koji/import (same volume).
> The ownership on the packages is daemon (same as /mnt/koji/) - not sure
> if this matters.
>
>
> sudo koji import --link /mnt/koji/alsa-lib-1.0.14-1.rc4.el5.src.rpm
>
> Linking rpm to:
> /mnt/koji/work/cli-import/1216842099.8955669.PLyYLFZS/alsa-lib-1.0.14-1.rc4.el5.src.rpm
>
>
> importing alsa-lib-1.0.14-1.rc4.el5.src.rpm... Fault: 'Traceback (most recent call last):\n File
> "/usr/share/koji-hub/kojixmlrpc.py", line 86, in _marshaled_dispatch\n
> response = self._dispatch(method, params)\n File
> "/usr/share/koji-hub/kojixmlrpc.py", line 156, in _dispatch\n ret =
> func(*params,**opts)\n File "/usr/share/koji-hub/kojihub.py", line 4110,
> in importRPM\n import_rpm_file(fn,rpminfo[\'build\'],rpminfo)\n File
> "/usr/share/koji-hub/kojihub.py", line 2991, in import_rpm_file\n
> os.rename(fn,final_path)\nOSError: [Errno 13] Permission denied\nLocals
> by frame, innermost last\nFrame HandlerDispatch in
> /usr/lib64/python2.4/site-packages/mod_python/importer.py at line 1537\n
> req = \n parent = None\n config =
> {\'PythonAutoReload\': \'0\', \'PythonDebug\': \'1\'}\n self =
> \n cache = WHILE PRINTING VALUE>\n default_handler = handler\n phase =
> PythonHandler\n handler = kojixmlrpc\n hlist =
> {\'handler\':\'kojixmlrpc\',\'directory\':\'/usr/share/koji-hub/\',\'silent\':0}\n
>
> aborted = False\n directory = /usr/share/koji-hub/\n root =
> /usr/share/koji-hub/\n options = {\'KojiTraceback\': \'extended\',
> \'KojiDebug\': \'On\', \'EmailDomain\': \'192.168.226.61\',
> \'KojiWebURL\': \'http://192.168.226.61/koji\
> ', \'LoginCreatesUser\': \'Off\',
> \'ProxyDNs\': \'/C=US/ST=New Jersey/O=Open System
> Solutions/CN=192.168.226.61\', \'DNUsernameComponent\': \'CN\',
> \'KojiDir\': \'/mnt/koji\', \'DBHost\': \'localhost\', \'DBUser\':
> \'koji\', \'DBName\': \'koji\'}\n result = 500\nFrame _process_target in
> /usr/lib64/python2.4/site-packages/mod_python/importer.py at line 1229\n
> req = \n silent = 0\n default =
> handler\n handler = kojixmlrpc\n object = 0x2b34997152a8>\n module = \'_mp_df1c13776110deb1e020020636703560\' from
> \'/usr/share/koji-hub/kojixmlrpc.py\'>\n directory =
> /usr/share/koji-hub/\n parts = [\'kojixmlrpc\']\n result = -1\n
> object_str = handler\n expected = [-1, 0]\n module_name = kojixmlrpc\n
> path = [\'/usr/share/koji-hub/\']\n config = {\'PythonAutoReload\':
> \'0\', \'PythonDebug\': \'1\'}\n arg = 0x2b3497cbd410>\nFrame _execute_target in
> /usr/lib64/python2.4/site-packages/mod_python/importer.py at line 1128\n
> one_process = False\n object = \n
> req = \n pdb_debug = 0\n arg =
> \n config = {\'PythonAutoReload\':
> \'0\', \'PythonDebug\': \'1\'}\nFrame handler in
> /usr/share/koji-hub/kojixmlrpc.py at line 291\n functions =
> <_mp_4e76479ee7b716cdbf29397025ee2356.RootExports object at
> 0x2b349c6f08d0>\n profiling = False\n h =
> <_mp_df1c13776110deb1e020020636703560.ModXMLRPCRequestHandler object at
> 0x2b349c6f0d10>\n req = \n
> hostFunctions = <_mp_4e76479ee7b716cdbf29397025ee2356.HostExports object
> at 0x2b349c6f0d90>\n opts = {\'KojiTraceback\': \'extended\',
> \'KojiDebug\': \'On\', \'EmailDomain\': \'192.168.226.61\',
> \'KojiWebURL\': \'http://192.168.226.61/koji\
> ', \'LoginCreatesUser\': \'Off\',
> \'ProxyDNs\': \'/C=US/ST=New Jersey/O=Open System
> Solutions/CN=192.168.226.61\', \'DNUsernameComponent\': \'CN\',
> \'KojiDir\': \'/mnt/koji\', \'DBHost\': \'localhost\', \'DBUser\':
> \'koji\', \'DBName\': \'koji\'}\nFrame handle_request in
> /usr/share/koji-hub/kojixmlrpc.py at line 242\n self =
> <_mp_df1c13776110deb1e020020636703560.ModXMLRPCRequestHandler object at
> 0x2b349c6f0d10>\n req = \nFrame
> _marshaled_dispatch in /usr/share/koji-hub/kojixmlrpc.py at line 112\n e
> = [Errno 13] Permission denied\n self =
> <_mp_df1c13776110deb1e020020636703560.ModXMLRPCRequestHandler object at
> 0x2b349c6f0d10>\n e_class = exceptions.OSError\n start = 1216842100.35\n
> faultCode = 1\n params = \n tb_str =
> Traceback (most recent call last):\n File
> "/usr/share/koji-hub/kojixmlrpc.py", line 86, in _marshaled_dispatch\n
> response = self._dispatch(method, params)\n File
> "/usr/share/koji-hub/kojixmlrpc.py", line 156, in _dispatch\n ret =
> func(*params,**opts)\n File "/usr/share/koji-hub/kojihub.py", line 4110,
> in importRPM\n import_rpm_file(fn,rpminfo[\'build\'],rpminfo)\n File
> "/usr/share/koji-hub/kojihub.py", line 2991, in import_rpm_file\n
> os.rename(fn,final_path)\nOSError: [Errno 13] Permission denied\n\n
> tb_type = extended\n data = version=\'1.0\'?>\n\nimportRPM\n\n\ncli-import/1216842099.8955669.PLyYLFZS\
Re: package upload/import speed
On Wed, 2008-07-23 at 15:56 -0400, Naveen Gavini wrote: > Hello all, > > When importing or building RPM's from source is there an option that is > throttling the upload speed of the package? > We are seeing speeds of 10kbps which makes importing/building large > packages very slow. Package upload happens via xml-rpc, and to avoid holding the entire contents of the file in memory, it's done in chunks. Unfortunately python xml-rpc libs don't support keep-alive, so if you're using SSL authentication you incur the non-trivial SSL connection setup/teardown overhead for every chunk, in addition to the TCP connection setup/teardown. The chunk size in your version is probably very small (64k?). That has been increased in the Koji repo, but not released yet. The next version will improve this situation somewhat. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: GenericError Repo directory missing
On Tue, 2008-07-22 at 17:19 -0400, Naveen Gavini wrote:
> Mike Bonnet wrote:
> > On Tue, 2008-07-22 at 16:02 -0400, Naveen Gavini wrote:
> >
> >> Hello all,
> >>
> >> We have installed the Koji build system and are able to import RPMS. We
> >> are having issues building from SRPMS as it's prompting us that it is
> >> unable to find the file once it is uploaded. The other issue is we have
> >> seen about 500 tasks fail for newRepo (noarch) with the message:
> >> GenericError: Repo directory missing:
> >> /mnt/koji/repos/rutgers-centos5-build/538.
> >>
> >
> > Does the /mnt/koji/repos directory exist? Is it owned by the same user
> > httpd runs as?
> >
> > Also, have you tried to change topdir to something other than /mnt/koji?
> > There are known problem with this, it needs to be fixed.
> >
> >
> We were able to fix this. The problem seemed to be that we did not have
> this directory (/mnt/koji) mounted on the builder, it was only mounted
> on the hub which runs kojira. We thought since kojira was creating the
> repos directories that would suffice. It also needs to be accessible by
> the builder.
It is possible to have a builder that doesn't need access to /mnt/koji
if you configure topurl, as Mike McLean suggested, and remove that
builder from the "createrepo" channel (so it won't take newRepo tasks).
However, you always need to have at least one builder that has
(read-only) access to /mnt/koji in the "createrepo" channel, or no repos
will ever get created/refreshed.
> Thanks.
> >> We have set the permissions to daemon on /mnt/koji (same user as apache).
> >>
> >> It seems when we try to manually look at the directory it exists, this
> >> is also the same for when the srpm is uploaded and it fails prompting us
> >> file not found, even though we can find it by looking at that directory
> >> on the server.
> >>
> >> Below is the kojira.log
> >>
> >> 2008-07-22 19:46:31,358 [DEBUG] koji.repo.manager: Reading current repo
> >> data
> >> 2008-07-22 19:46:36,855 [DEBUG] koji.repo.manager: Repo data:
> >> [{'create_event': 552, 'state': 0, 'tag_id': 2, 'tag_name':
> >> 'rutgers-centos5-build', 'create_ts': 1216754980.8750801, 'id': 534},
> >> {'create_event': 553, 'state': 0, 'tag_id': 2, 'tag_name':
> >> 'rutgers-centos5-build', 'create_ts': 1216755215.82616, 'id': 535}]
> >> 2008-07-22 19:46:42,399 [DEBUG] koji.repo.manager: Needed tags: [2]
> >> 2008-07-22 19:46:42,400 [DEBUG] koji.repo.manager: Current tags: [2]
> >> 2008-07-22 19:46:42,400 [DEBUG] koji.repo: Checking for changes: [1, 2]
> >> 2008-07-22 19:46:47,902 [DEBUG] koji.repo: No tag changes since event 552
> >> 2008-07-22 19:46:47,902 [DEBUG] koji.repo: Checking for changes: [1, 2]
> >> 2008-07-22 19:46:53,429 [DEBUG] koji.repo: No tag changes since event 553
> >> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: order: []
> >> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: repo 534: tag=2,
> >> state=INIT
> >> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: repo 535: tag=2,
> >> state=INIT
> >> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: task 548 for tag 2
> >> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: Scanning filesystem
> >> for repos
> >> 2008-07-22 19:47:09,590 [INFO] koji.repo.manager: Problem: newRepo task
> >> 548 for tag 2 is FAILED
> >> 2008-07-22 19:47:09,591 [DEBUG] koji.repo.manager: Current tasks: {}
> >> 2008-07-22 19:47:09,591 [DEBUG] koji.repo.manager: Updating repos
> >> 2008-07-22 19:47:09,591 [DEBUG] koji.repo.manager: Reading current repo
> >> data
> >> 2008-07-22 19:47:15,273 [DEBUG] koji.repo.manager: Repo data:
> >> [{'create_event': 552, 'state': 0, 'tag_id': 2, 'tag_name':
> >> 'rutgers-centos5-build', 'create_ts': 1216754980.8750801, 'id': 534},
> >> {'create_event': 553, 'state': 0, 'tag_id': 2, 'tag_name':
> >> 'rutgers-centos5-build', 'create_ts': 1216755215.82616, 'id': 535},
> >> {'create_event': 554, 'state': 0, 'tag_id': 2, 'tag_name':
> >> 'rutgers-centos5-build', 'create_ts': 1216756145.8847401, 'id': 536}]
> >> 2008-07-22 19:47:15,273 [INFO] koji.repo.manager: Found repo 536,
> >> state=INIT
> >> 2008-07-22 19:47:26,421 [DEBUG] koji.repo.manager: Needed tags: [2]
> >> 2008-07-22 19:47:26,421 [DEBUG] koji.repo.manager: Current tags: [2]
> >> 2008-07-22 19:47:26,421 [DEBUG] koji.repo: Checking for changes: [1, 2]
> >> 2008-07-22 19:47:31,905 [DEBUG] koji.repo: No tag changes since event 554
> >> 2008-07-22 19:47:31,905 [DEBUG] koji.repo: Checking for changes: [1, 2]
> >>
> >> Any idea?
> >>
> >> Thanks.
> >>
> >>
> >
> > --
> > Fedora-buildsys-list mailing list
> > Fedora-buildsys-list@redhat.com
> > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
> >
>
>
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: GenericError Repo directory missing
On Tue, 2008-07-22 at 16:02 -0400, Naveen Gavini wrote:
> Hello all,
>
> We have installed the Koji build system and are able to import RPMS. We
> are having issues building from SRPMS as it's prompting us that it is
> unable to find the file once it is uploaded. The other issue is we have
> seen about 500 tasks fail for newRepo (noarch) with the message:
> GenericError: Repo directory missing:
> /mnt/koji/repos/rutgers-centos5-build/538.
Does the /mnt/koji/repos directory exist? Is it owned by the same user
httpd runs as?
Also, have you tried to change topdir to something other than /mnt/koji?
There are known problem with this, it needs to be fixed.
> We have set the permissions to daemon on /mnt/koji (same user as apache).
>
> It seems when we try to manually look at the directory it exists, this
> is also the same for when the srpm is uploaded and it fails prompting us
> file not found, even though we can find it by looking at that directory
> on the server.
>
> Below is the kojira.log
>
> 2008-07-22 19:46:31,358 [DEBUG] koji.repo.manager: Reading current repo data
> 2008-07-22 19:46:36,855 [DEBUG] koji.repo.manager: Repo data:
> [{'create_event': 552, 'state': 0, 'tag_id': 2, 'tag_name':
> 'rutgers-centos5-build', 'create_ts': 1216754980.8750801, 'id': 534},
> {'create_event': 553, 'state': 0, 'tag_id': 2, 'tag_name':
> 'rutgers-centos5-build', 'create_ts': 1216755215.82616, 'id': 535}]
> 2008-07-22 19:46:42,399 [DEBUG] koji.repo.manager: Needed tags: [2]
> 2008-07-22 19:46:42,400 [DEBUG] koji.repo.manager: Current tags: [2]
> 2008-07-22 19:46:42,400 [DEBUG] koji.repo: Checking for changes: [1, 2]
> 2008-07-22 19:46:47,902 [DEBUG] koji.repo: No tag changes since event 552
> 2008-07-22 19:46:47,902 [DEBUG] koji.repo: Checking for changes: [1, 2]
> 2008-07-22 19:46:53,429 [DEBUG] koji.repo: No tag changes since event 553
> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: order: []
> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: repo 534: tag=2,
> state=INIT
> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: repo 535: tag=2,
> state=INIT
> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: task 548 for tag 2
> 2008-07-22 19:46:53,430 [DEBUG] koji.repo.manager: Scanning filesystem
> for repos
> 2008-07-22 19:47:09,590 [INFO] koji.repo.manager: Problem: newRepo task
> 548 for tag 2 is FAILED
> 2008-07-22 19:47:09,591 [DEBUG] koji.repo.manager: Current tasks: {}
> 2008-07-22 19:47:09,591 [DEBUG] koji.repo.manager: Updating repos
> 2008-07-22 19:47:09,591 [DEBUG] koji.repo.manager: Reading current repo data
> 2008-07-22 19:47:15,273 [DEBUG] koji.repo.manager: Repo data:
> [{'create_event': 552, 'state': 0, 'tag_id': 2, 'tag_name':
> 'rutgers-centos5-build', 'create_ts': 1216754980.8750801, 'id': 534},
> {'create_event': 553, 'state': 0, 'tag_id': 2, 'tag_name':
> 'rutgers-centos5-build', 'create_ts': 1216755215.82616, 'id': 535},
> {'create_event': 554, 'state': 0, 'tag_id': 2, 'tag_name':
> 'rutgers-centos5-build', 'create_ts': 1216756145.8847401, 'id': 536}]
> 2008-07-22 19:47:15,273 [INFO] koji.repo.manager: Found repo 536, state=INIT
> 2008-07-22 19:47:26,421 [DEBUG] koji.repo.manager: Needed tags: [2]
> 2008-07-22 19:47:26,421 [DEBUG] koji.repo.manager: Current tags: [2]
> 2008-07-22 19:47:26,421 [DEBUG] koji.repo: Checking for changes: [1, 2]
> 2008-07-22 19:47:31,905 [DEBUG] koji.repo: No tag changes since event 554
> 2008-07-22 19:47:31,905 [DEBUG] koji.repo: Checking for changes: [1, 2]
>
> Any idea?
>
> Thanks.
>
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Supporting EPEL Builds in Koji
On Thu, 2008-07-17 at 13:54 -0400, Mike McLean wrote: > Mike Bonnet wrote: > > http://fedoraproject.org/wiki/Koji/EPELSupport > > This is mostly in line with what I've been thinking. I do have a few > comments/concerns thought... > > If the remote_repo_url data is going to be inherited (and I tend to > think it should be), then I think it should be in a separate table. I'd > like to reserve tag_config for data that is local to individual tags. > This will also make it easier to represent multiple remote repos. I don't have any problem with this, though it does mean we'll need to duplicate quite a bit of the inheritance-walking code, or make it configurable as to which inheritance it's walking. This new table would also have to be versioned, the same way the tag_config table is. > I'm a little concerned about using the rpminfo table. Yes, I know it > seems wasteful to introduce another table to track very similar data, > but these remote rpms really are differently tracked and handled than > the local ones. The big win here is that the methods and tools that query rpminfo for information about what was present in the buildroot at build time wouldn't have to change, or only change slightly. With minor modification the web UI can continue to show a list of all packages in a buildroot, along with a flag indicating if they were local or remote. The buildroot_listing table would not have to change at all. The majority of XML-RPC calls that interact with the rpminfo or buildroot_listing tables would only need minor modifications. Adding a new table to track remote rpms metadata and which remote rpms end up in a buildroot would add significant effort to this proposal. Also, I think it's more semantically correct to have a single place where we track rpm metadata and buildroot contents, regardless of where they came from. > Also, I'm not sure how I feel about having rpminfo entries will null > build_id. Sure, technically the field lacks the 'not null' constraint, > but that is more of an oversight. Yes, I realize that the "not null" constraint should exist now, and in fact all rpms in the Fedora database do reference builds. However, I think logically having a remote rpm not reference a local build makes sense. The alternative is to create the build object from the srpm info in the repodata (along with some namespacing similar to rpminfo). However, this would significantly clutter the build table with information that is pretty non-essential. > Note, I'm not outright rejecting the idea of using rpminfo this way, but > I am concerned. > > > As for the origin field. I think we should track where these external > rpms come from, but I'm not sure about including in the uniqueness > constraint. I'm not sure that the value of that field is sufficiently > well defined (or canonicalizable) for such use. I'd rather see the > sigmd5 value (or some abstracting sighash field) used as a unique index. I'm open to suggestions on how to modify the uniqueness constraint to handle this case. We care about ensuring that a locally-built rpm doesn't have the same n-v-r as another locally-built rpm. I don't think we care at all about n-v-r uniqueness amongst remote rpms. However, we probably want to avoid creating 2 rpminfo entries when the same remote rpm is used in 2 different buildroots. Using the sigmd5 is a good way to avoid that. However, what happens if a remote rpm with the same n-v-r and sigmd5 gets pulled in from 2 different remote repos? Perhaps the "origin" field should be pushed down to the buildroot_listing table, so the buildroots can reference the same rpminfo object, but indicate that it came from a different repo in each buildroot? Also, what happens when we find 2 remote rpms with the same n-v-r but different sigmd5s? Should that be an error? > Following are additional ideas relating to this feature. They are > perhaps a bit ambitious for the short term, but I'd at least like to > keep them in mind with the initial design so we don't paint ourselves > into a corner. > > First, I'd like to be able to support external koji servers (or rather a > target or tag from an external koji server) in addition to external > repos. Some of the ideas are the same, however an external koji server > provides more information and more structure. I agree that this is a desirable goal. I believe this is more the domain of the Koji secondary-arch daemon. It would be talking directly to an "upstream" Koji server, analyzing what it's doing, and applying some logic to decide what builds to import or replicate, and where/how to do it. This proposal has the much more modest goal of simply consuming static external repos, and is more appropriate for the
Re: mod_python error installing Koji
On Wed, 2008-07-16 at 16:53 -0400, Naveen Gavini wrote:
> Hello all,
>
> We are trying to setup the Koji build system for our Centos and Fedora
> repositories. We are getting the errors below after following the setup
> guide.
> We have tried numerous different things to attempt to correct the errors
> and nothing has worked. We initially thought it was an issue of what
> user it was
> being run as and we changed users around this did not work. We changed
> various settings in our apache configuration and still no dice.
> Here is the errors we are seeing on http://192.168.226.61/koji/:
>
> MOD_PYTHON ERROR
>
> ProcessId: 9453
> Interpreter:'127.0.0.1'
>
> ServerName: '127.0.0.1'
> DocumentRoot: '/var/www/html'
>
> URI:'/koji/'
> Location: None
> Directory: '/usr/share/koji-web/scripts/'
> Filename: '/usr/share/koji-web/scripts/index.py'
> PathInfo: ''
>
> Phase: 'PythonHandler'
> Handler:'mod_python.publisher'
>
> Traceback (most recent call last):
>
> File "/usr/lib64/python2.4/site-packages/mod_python/importer.py", line
> 1537, in HandlerDispatch
> default=default_handler, arg=req, silent=hlist.silent)
>
> File "/usr/lib64/python2.4/site-packages/mod_python/importer.py", line
> 1229, in _process_target
> result = _execute_target(config, req, object, arg)
>
> File "/usr/lib64/python2.4/site-packages/mod_python/importer.py", line
> 1128, in _execute_target
> result = object(arg)
>
> File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py", line
> 213, in handler
> published = publish_object(req, object)
>
> File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py", line
> 425, in publish_object
> return publish_object(req,util.apply_fs_data(object, req.form, req=req))
>
> File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line 554, in
> apply_fs_data
> return object(**args)
>
> File "/usr/share/koji-web/scripts/index.py", line 175, in index
> start=buildStart, dataName='builds', prefix='build', order=buildOrder,
> pageSize=10)
>
> File "/usr/share/koji-web/lib/kojiweb/util.py", line 109, in paginateMethod
> totalRows = getattr(server, methodName)(*args, **kw)
>
> File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1077, in
> __call__
> return self.__func(self.__name,args,opts)
>
> File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1302, in
> _callMethod
> return proxy.__getattr__(name)(*args)
>
> File "/usr/lib64/python2.4/xmlrpclib.py", line 1096, in __call__
> return self.__send(self.__name, args)
>
> File "/usr/lib64/python2.4/xmlrpclib.py", line 1383, in __request
> verbose=self.__verbose
>
> File "/usr/lib64/python2.4/xmlrpclib.py", line 1137, in request
> headers
>
> ProtocolError: Error>
>
>
> MODULE CACHE DETAILS
>
> Accessed: Wed Jul 16 16:47:56 2008
> Generation: 1
>
> _mp_0dec3ca8c086f5baed01b0d5504fa2b0 {
> FileName: '/usr/share/koji-web/scripts/index.py'
> Instance: 1
> Generation: 1
> Modified: Fri Dec 14 21:12:36 2007
> Imported: Wed Jul 16 16:36:02 2008
> }
>
>
> Here is the error we are seeing on http://192.168.226.61/koji/index.chtml:
> Forbidden
> You don't have permission to access /koji/index.chtml on this server.
>
> Here is the error we are seeing on http://192.168.226.61/kojihub:
> Internal Server Error
> blah blah blah
>
> http://192.168.226.61/koji-static/
> Displays a directory listing of a few files and directories so I am
> assuming it is working correctly.
You should see more detailed error messages in /var/log/httpd/error_log
(or ssl_error_log, depending on your setup). I'm guessing the "apache"
OS user does not have permission to connect to the "koji" database as
the "koji" database user. You need to grant the appropriate access in
pg_hba.conf.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji CLI Auth problem
On Wed, 2008-07-16 at 11:06 +0800, Linul wrote:
> HI:
>
> I'm using CentOS 5.2 for my Koji Server, but now I have a problem
> about Koji CLI auth.
>
> According the wiki document in
> http://fedoraproject.org/wiki/Koji/ServerHowTo , I setup my Koji-hub、
> Koji-web、postgresql
>
> , and have a koji web interface.
>
> I also setup my CA Center，and configure the kojiweb.conf、
> kojihub.conf、/etc/koji.conf.
>
> But when i execute the koji command with no username and password, the
> messages is：
>
> Error: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL
> routines', 'SSL_CTX_use_PrivateKey_file', 'PEM lib')]
Your client certificate file (indicated by "cert" in the config file)
needs to contain both the certificate and private key. Your private key
is missing.
> why?
>
> thanks.
>
>
> /etc/koji.conf：
>
> [koji]
>
> ;configuration for koji cli tool
>
> ;url of XMLRPC server
> ;server = http://koji.fedoraproject.org/kojihub
> server = http://koji.ossii.com.tw/kojihub
>
> ;url of web interface
> ;weburl = http://koji.fedoraproject.org/koji
> weburl = http://koji.ossii.com.tw/koji
>
> ;url of package download site
> ;pkgurl = http://koji.fedoraproject.org/packages
> pkgurl = http://koji.ossii.com.tw/packages
>
> ;path to the koji top directory
> topdir = /mnt/koji
>
> ;configuration for SSL athentication
>
> ;client certificate
> ;cert = ~/.fedora.cert
> cert = /etc/kojid/kojiadmin.crt
>
> ;certificate of the CA that issued the client certificate
> ;ca = ~/.fedora-upload-ca.cert
> ca = /etc/kojid/kojiadmin.key
>
> ;certificate of the CA that issued the HTTP server certificate
> ;serverca = ~/.fedora-server-ca.cert
> serverca = /etc/httpd/conf.d/ssl/ossiikojica.crt
>
>
> kojihub.conf：
>
>
> SetHandler mod_python
> PythonHandler kojixmlrpc
> PythonOption DBName koji
> PythonOption DBUser kevin
> PythonOption DBHost 127.0.0.1
> PythonOption KojiDir /mnt/koji
>
> # Kerberos auth configuration
> # PythonOption AuthPrincipal [EMAIL PROTECTED]
> # PythonOption AuthKeytab /etc/koji.keytab
> # PythonOption ProxyPrincipals [EMAIL PROTECTED]
> # format string for host principals (%s = hostname)
> # PythonOption HostPrincipalFormat compile/[EMAIL PROTECTED]
> # end Kerberos auth configuration
>
> # SSL client certificate auth configuration
> # the client username is the common name of the subject of
> their client certificate
> PythonOption DNUsernameComponent CN
> # separate multiple DNs with |
> # PythonOption ProxyDNs "/C=US/ST=Massachusetts/O=Example
> Org/OU=Example User/CN=example/[EMAIL PROTECTED]"
> PythonOption ProxyDNs "/C=TW/ST=Taiwan/O=OSSII/OU=Koji Hub
> Server/CN=OSSII Koji Server CA/[EMAIL PROTECTED]"
> # end SSL client certificate auth configuration
>
> PythonOption LoginCreatesUser On
> PythonOption KojiWebURL http://koji.ossii.com.tw/koji
>
> # The domain name that will be appended to Koji usernames
> # when creating email notifications
> PythonOption EmailDomain example.com
> # PythonOption KojiDebug On
> # PythonOption KojiTraceback "extended"
> # sending tracebacks to the client isn't very helpful for
> debugging xmlrpc
> PythonDebug Off
> # autoreload is mostly useless to us (it would only reload
> kojixmlrpc.py)
> PythonAutoReload Off
>
>
> # uncomment this to enable authentication via SSL client certificates
>
> SSLOptions +StdEnvVars
>
> # these options must be enabled globally (in ssl.conf)
> SSLVerifyClient require
> SSLVerifyDepth 10
>
> kojiweb.conf：
>
> Alias /koji "/usr/share/koji-web/scripts/"
>
>
> # Config for the publisher handler
> SetHandler mod_python
> PythonHandler mod_python.publisher
>
> # General settings
> PythonDebug On
> PythonOption KojiHubURL http://koji.ossii.com.tw/kojihub
> PythonOption KojiWebURL http://koji.ossii.com.tw/koji
> PythonOption KojiPackagesURL
> http://koji.ossii.com.tw/koji/packages
> PythonOption WebPrincipal koji/[EMAIL PROTECTED]
> PythonOption WebKeytab /etc/httpd.keytab
> PythonOption WebCCache /var/tmp/kojiweb.ccache
> PythonOption WebCert /etc/httpd/conf.d/ssl/kojiweb.crt
> PythonOption ClientCA /etc/httpd/conf.d/ssl/kojiweb.key
> PythonOption KojiHubCA /etc/httpd/conf.d/ssl/ossiikojica.crt
> PythonOption LoginTimeout 72
> # This must be changed before deployment
> PythonOption Secret CHANGE_ME
> PythonPath "sys.path + ['/usr/share/koji-web/lib']"
> PythonCleanupHandler kojiweb.handlers::cleanup
> PythonAutoReload Off
>
>
> SSLOptions +StdEnvVars
>
> # these options must be enabled globally (in ssl.conf)
> SSLVerifyClient require
> SSLVerifyDepth 10
>
> Alias /koji-static/ "/usr/share/koji-web/static/"
>
>
> Options None
> AllowOverride None
> Order
Re: Supporting EPEL Builds in Koji
On Thu, 2008-07-10 at 19:12 +0200, Jeroen van Meeuwen wrote: > Mike Bonnet wrote: > > Hi. I've written up a proposal for a way to support EPEL builds in > > Koji. It's not the only way we could do this, but I think it's doable > > with a reasonable amount of effort, and has the side-effect of greatly > > simplifying the Koji setup process for a lot of people (by removing the > > need to bootstrap/import an entire distro of packages into your private > > Koji instance). You can view the proposal here: > > > > http://fedoraproject.org/wiki/Koji/EPELSupport > > > > It's fairly detailed regarding the data model changes necessary, so if > > you're not familiar with the Koji codebase you can skip those parts. > > Questions and comments welcome. > > > > Hi Mike, > > good to see you've spend some time on this whereas I have been lazy in > Littleton (holiday). > > I'd like to share a few thoughts on the Wiki page -which is a great start; > > From the Wiki page: "There is a strong feeling that if a package exists > in the Koji-managed local repo (whose contents the Koji admin has full > control over) it should always be preferred over the external repo > (whose contents the Koji admin may have little or no control over)." > > The preference koji will have (in using which package in the buildroot), > might introduce the problem where customly built package foo-1.0 is used > in the buildroot, and upstream updates to foo-1.1 - the running nodes > would update to foo-1.1 whereas the buildroot still uses the custom > foo-1.0... Yes, it's up to the Koji admin to monitor the remote repo, and take appropriate action when their custom local packages are superseded by packages in the remote repo. That may be untagging or blocking the package locally so the newer version can be pulled down from the remote repo. Or it may be rebuilding the custom package based on the updated sources. The point is that the build environment doesn't change unless the Koji admin takes some action to change it. > The point being, that these updates have to managed as they are > released. The updates need to managed on the side where said packages > are being mashed into a repository (infra side) or applied (client side). > > You can see the duplicate effort when the updates are managed on either > side (infra or client), _and_ in koji, separately. There is duplicate effort either way. The difference is that, if highest-nvr-wins is used, and a remote repo updates to a later version of a package that you have a custom build of, there is *no way* for you to revert your build environment to that lower-nvr version without bumping your version higher than their version (without actually changing the source at all) and rebuilding. It encourages this Cold War arms-race of version numbers between your custom packages and the remote repo's packages, and results in the admin having to fake higher version numbers and rebuild constantly *without any source changes* just to keep their custom packages in their build environment. Alternately, if first-match-wins is used (where the first repo is the locally-managed Koji repo), and a remote repo updates to a later version of a package you have a custom version of, nothing happens to your build environment. If you decide you want the newer version from the remote repo, you untag your local package and let it get pulled in from the remote repo. If that newer version has problems, retag your custom version and it will then be available in the build environment again. There is no unnecessary building of packages, no faking version numbers, and no unexpected changes to your build environment. It's the "principle of least surprise", which is why I think it's the right policy to use in a managed build environment like Koji. > I would like to suggest the koji development team makes the priority > setting koji is going to use a configurable item -which in compared to > the bigger picture isn't all that much a priority, just something to > think about. I strongly feel that this isn't something that needs to be configurable, and that first-match-wins is the correct behavior. But if other people agree that there is a valid use-case for making it configurable, and Seth and/or James can make the logic in repomerge configurable, then we can add switch for it to Koji. > Additionally, I'd like to comment on / ask about the proposed database > changes for the tag_config table; In an attempt to show you what I was > thinking, here's a number of questions; > > From the Wiki page: "At repo creation time, the repodata will be > retrieved from the processed url and merged with the local repodata as > de
Supporting EPEL Builds in Koji
Hi. I've written up a proposal for a way to support EPEL builds in Koji. It's not the only way we could do this, but I think it's doable with a reasonable amount of effort, and has the side-effect of greatly simplifying the Koji setup process for a lot of people (by removing the need to bootstrap/import an entire distro of packages into your private Koji instance). You can view the proposal here: http://fedoraproject.org/wiki/Koji/EPELSupport It's fairly detailed regarding the data model changes necessary, so if you're not familiar with the Koji codebase you can skip those parts. Questions and comments welcome. Thanks, Mike -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: koji add-group?
On Thu, 2008-05-08 at 16:09 -0500, Paul B Schroeder wrote:
> Hmm.. Strange.. I don't seem to have the add-group command (see
> further below). I had to do this via psql to be able to "koji
> add-group-pkg":
>
> koji=> insert into groups (name) values('build');
> INSERT 0 1
> koji=> select * from groups;
> id | name
> +---
> 1 | build
> (1 row)
> koji=> insert into group_config (group_id, tag_id, display_name)
> values(1, 2, 'build');
> INSERT 0 1
> koji=> select * from group_config;
> group_id | tag_id | blocked | exported | display_name | is_default |
> uservisible | description | langonly | biarchonly | create_event |
> revoke_event | active
> --++-+--+--++-+-+--++--+--+
> 1 | 2 | f | t| build||
> | | || 3413 | | t
> (1 row)
>
> koji=> select * from group_package_listing;
> group_id | tag_id | package | blocked | type | basearchonly |
> requires | create_event | revoke_event | active
> --++-+-+-+--+--+--+--+
> 1 | 2 | bash| f | default | |
> | 3414 | | t
> (1 row)
>
>
> --
>
>
> [EMAIL PROTECTED] koji]# rpm -q koji
> koji-1.2.2-2.fc8
>
> [EMAIL PROTECTED] koji]# koji add-group --help
> Available commands:
> buildBuild a package from source
> buildinfoPrint basic information about a build
> cancel Cancel tasks and/or builds
> chain-build Build one or more packages from source
> download-build Download a built package
> help List available commands
> latest-pkg Print the latest packages for a tag
> list-api Print the list of XML-RPC APIs
> list-buildroot List the rpms used in or built in a
> buildroot
> list-groups Print the group listings
> list-hosts Print the host listing
> list-pkgsPrint the package listing for tag or for
> owner
> list-tag-history Print a history of tag operations
> list-tag-inheritance Print the inheritance information for a tag
> list-tagged List the builds or rpms in a tag
> list-tagsPrint the list of tags
> list-targets List the build targets
> list-tasks Print the list of tasks
> list-untaggedList untagged builds
> mock-config Create a mock config
> move-pkg 'Move' one or more packages between tags
> resubmit Retry a canceled or failed task, using the
> same parameter as the original task.
> rpminfo Print basic information about an RPM
> show-groups Show groups data for a tag
> tag-pkg Apply a tag to one or more packages
> taginfo Print basic information about a tag
> taskinfo Show information about a task
> untag-pkgRemove a tag from one or more packages
> watch-logs Watch logs in realtime
> watch-task Track progress of particular tasks
> (Type "koji --help" for help about global options
> or "koji --help" for help about a particular command's
> options.)
> Usage: koji [global-options] command [command-options-and-arguments]
>
> koji: error: Unknown command: add_group
That must have been added to koji-1.2.3. You can grab it from
http://koji.fedoraproject.org/koji/buildinfo?buildID=28165 . It should
work fine on F8.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: koji.build: Session expired
On Wed, 2008-05-07 at 15:34 -0500, Paul B Schroeder wrote:
> Okay.. I *thought* I had it working, but now it's not. At the very
> least, kojid doesn't appear to be shutting down anymore. Running kojira
> with --verbose and --debug doesn't appear to be of much help. I see
> these lines in the log file when starting kojira:
>
> 2008-05-07 10:28:24,540 [DEBUG] koji.repo.manager: Reading current repo
> data
> 2008-05-07 10:28:24,678 [DEBUG] koji.repo.manager: Repo data: []
> 2008-05-07 10:28:24,678 [DEBUG] koji.repo.manager: Scanning filesystem
> for repos
>
> After starting, I then check to see if the process is running. But it's
> not.. Any more clues?
Try running it with --fg and see if you can get a traceback.
> Thanks...Paul...
>
>
>
> On Wed, 2008-05-07 at 12:46 -0400, Mike Bonnet wrote:
> > On Wed, 2008-05-07 at 11:27 -0500, Paul B Schroeder wrote:
> > > Oh yea.. It should also be noted that I set this up using SSL
> > > authentication, not kerberos..
> > >
> > > Thanks again...Paul..
> >
> > Are kojid and kojira using the same SSL cert for authentication? This
> > would cause the problem you're seeing, as they would be sharing the same
> > session, and when one of them logs out, the other would think it had
> > been logged out as well. You need to use different SSL certs (with
> > different CNs) for each user, including the daemons.
> >
> > >
> > > On Wed, 2008-05-07 at 11:24 -0500, Paul B Schroeder wrote:
> > > > Hello..
> > > >
> > > > I'm running F8 with the latest koji-* packages:
> > > > [EMAIL PROTECTED] koji]# rpm -qa 'koji*'
> > > > koji-hub-1.2.2-2.fc8
> > > > koji-1.2.2-2.fc8
> > > > koji-utils-1.2.2-2.fc8
> > > > koji-web-1.2.2-2.fc8
> > > > koji-builder-1.2.2-2.fc8
> > > >
> > > > I've been following the Koji Server HOWTO here:
> > > > http://fedoraproject.org/wiki/Koji/ServerHowTo
> > > >
> > > > I've gotten all of the way through it, but I'm having issues when
> > > > starting kojira. It doesn't stay running and it seems to somehow be
> > > > causing kojid to shutdown as well. I've turned on debug for kojid and
> > > > kojira. I see this in kojid.log:
> > > >
> > > > 2008-05-07 06:10:37,895 [ERROR] koji.build: Session expired
> > > > 2008-05-07 06:10:37,896 [WARNING] koji.build: Shutting down, please
> > > > wait...
> > > >
> > > > Below is a full output of what I see in those logs. Am I missing
> > > > something simple? Anybody have any clue what's going wrong?
> > > >
> > > > Thanks in advance...Paul...
> > > >
> > > >
> > > > /var/log/kojid.log (before I attempt to start kojira):
> > > > 2008-05-07 06:10:04,662 [INFO] koji.build: Starting up
> > > > 2008-05-07 06:10:05,125 [DEBUG] koji.build.TaskManager: Local
> > > > buildroots: 0
> > > > 2008-05-07 06:10:05,125 [DEBUG] koji.build.TaskManager: Active
> > > > buildroots: 0
> > > > 2008-05-07 06:10:05,126 [DEBUG] koji.build.TaskManager: Expired/stray
> > > > buildroots: 0
> > > > 2008-05-07 06:10:05,438 [DEBUG] koji.build.TaskManager: Task Load: 0.0
> > > > 2008-05-07 06:10:05,438 [DEBUG] koji.build.TaskManager: Current tasks:
> > > > {}
> > > > 2008-05-07 06:10:05,743 [DEBUG] koji.build.TaskManager: hostdata:
> > > > {'capacity': 2.0, 'name': 'koji.bcstx.cacheflow.com', 'enabled': True,
> > > > 'arches': 'i386 x86_64', 'task_load': 0.0, 'ready': True, 'user_id': 3,
> > > > 'id': 1}
> > > > 2008-05-07 06:10:05,750 [DEBUG] koji.build.TaskManager: disk space
> > > > available in '/var/lib/mock': 43829 MB
> > > > 2008-05-07 06:10:06,186 [DEBUG] koji.build.TaskManager: Load Data:
> > > > 2008-05-07 06:10:06,186 [DEBUG] koji.build.TaskManager: hosts:
> > > > [{'capacity': 2.0, 'name': 'koji.bcstx.cacheflow.com', 'channels': [1,
> > > > 2], 'arches': 'i386 x86_64', 'task_load': 0.0, 'id': 1}]
> > > > 2008-05-07 06:10:06,187 [DEBUG] koji.build.TaskManager: tasks: []
> > > > 2008-05-07 06:10:06,187 [DEBUG] koji.build.TaskManager: bins:
> > > > {'1:noarch': 1, '2:noarch
Re: koji.build: Session expired
On Wed, 2008-05-07 at 11:27 -0500, Paul B Schroeder wrote:
> Oh yea.. It should also be noted that I set this up using SSL
> authentication, not kerberos..
>
> Thanks again...Paul..
Are kojid and kojira using the same SSL cert for authentication? This
would cause the problem you're seeing, as they would be sharing the same
session, and when one of them logs out, the other would think it had
been logged out as well. You need to use different SSL certs (with
different CNs) for each user, including the daemons.
>
> On Wed, 2008-05-07 at 11:24 -0500, Paul B Schroeder wrote:
> > Hello..
> >
> > I'm running F8 with the latest koji-* packages:
> > [EMAIL PROTECTED] koji]# rpm -qa 'koji*'
> > koji-hub-1.2.2-2.fc8
> > koji-1.2.2-2.fc8
> > koji-utils-1.2.2-2.fc8
> > koji-web-1.2.2-2.fc8
> > koji-builder-1.2.2-2.fc8
> >
> > I've been following the Koji Server HOWTO here:
> > http://fedoraproject.org/wiki/Koji/ServerHowTo
> >
> > I've gotten all of the way through it, but I'm having issues when
> > starting kojira. It doesn't stay running and it seems to somehow be
> > causing kojid to shutdown as well. I've turned on debug for kojid and
> > kojira. I see this in kojid.log:
> >
> > 2008-05-07 06:10:37,895 [ERROR] koji.build: Session expired
> > 2008-05-07 06:10:37,896 [WARNING] koji.build: Shutting down, please
> > wait...
> >
> > Below is a full output of what I see in those logs. Am I missing
> > something simple? Anybody have any clue what's going wrong?
> >
> > Thanks in advance...Paul...
> >
> >
> > /var/log/kojid.log (before I attempt to start kojira):
> > 2008-05-07 06:10:04,662 [INFO] koji.build: Starting up
> > 2008-05-07 06:10:05,125 [DEBUG] koji.build.TaskManager: Local
> > buildroots: 0
> > 2008-05-07 06:10:05,125 [DEBUG] koji.build.TaskManager: Active
> > buildroots: 0
> > 2008-05-07 06:10:05,126 [DEBUG] koji.build.TaskManager: Expired/stray
> > buildroots: 0
> > 2008-05-07 06:10:05,438 [DEBUG] koji.build.TaskManager: Task Load: 0.0
> > 2008-05-07 06:10:05,438 [DEBUG] koji.build.TaskManager: Current tasks:
> > {}
> > 2008-05-07 06:10:05,743 [DEBUG] koji.build.TaskManager: hostdata:
> > {'capacity': 2.0, 'name': 'koji.bcstx.cacheflow.com', 'enabled': True,
> > 'arches': 'i386 x86_64', 'task_load': 0.0, 'ready': True, 'user_id': 3,
> > 'id': 1}
> > 2008-05-07 06:10:05,750 [DEBUG] koji.build.TaskManager: disk space
> > available in '/var/lib/mock': 43829 MB
> > 2008-05-07 06:10:06,186 [DEBUG] koji.build.TaskManager: Load Data:
> > 2008-05-07 06:10:06,186 [DEBUG] koji.build.TaskManager: hosts:
> > [{'capacity': 2.0, 'name': 'koji.bcstx.cacheflow.com', 'channels': [1,
> > 2], 'arches': 'i386 x86_64', 'task_load': 0.0, 'id': 1}]
> > 2008-05-07 06:10:06,187 [DEBUG] koji.build.TaskManager: tasks: []
> > 2008-05-07 06:10:06,187 [DEBUG] koji.build.TaskManager: bins:
> > {'1:noarch': 1, '2:noarch': 1, '1:x86_64': 1, '1:i386': 1, '2:x86_64':
> > 1, '2:i386': 1}
> >
> > /var/log/kojid.log (after I start kojira):
> > 2008-05-07 06:10:21,504 [DEBUG] koji.build.TaskManager: Local
> > buildroots: 0
> > 2008-05-07 06:10:21,505 [DEBUG] koji.build.TaskManager: Active
> > buildroots: 0
> > 2008-05-07 06:10:21,505 [DEBUG] koji.build.TaskManager: Expired/stray
> > buildroots: 0
> > 2008-05-07 06:10:21,818 [DEBUG] koji.build.TaskManager: Task Load: 0.0
> > 2008-05-07 06:10:21,818 [DEBUG] koji.build.TaskManager: Current tasks:
> > {}
> > 2008-05-07 06:10:22,138 [DEBUG] koji.build.TaskManager: hostdata:
> > {'capacity': 2.0, 'name': 'koji.bcstx.cacheflow.com', 'enabled': True,
> > 'arches': 'i386 x86_64', 'task_load': 0.0, 'ready': True, 'user_id': 3,
> > 'id': 1}
> > 2008-05-07 06:10:22,145 [DEBUG] koji.build.TaskManager: disk space
> > available in '/var/lib/mock': 43829 MB
> > 2008-05-07 06:10:22,586 [DEBUG] koji.build.TaskManager: Load Data:
> > 2008-05-07 06:10:22,586 [DEBUG] koji.build.TaskManager: hosts:
> > [{'capacity': 2.0, 'name': 'koji.bcstx.cacheflow.com', 'channels': [1,
> > 2], 'arches': 'i386 x86_64', 'task_load': 0.0, 'id': 1}]
> > 2008-05-07 06:10:22,587 [DEBUG] koji.build.TaskManager: tasks: []
> > 2008-05-07 06:10:22,587 [DEBUG] koji.build.TaskManager: bins:
> > {'1:noarch': 1, '2:noarch': 1, '1:x86_64': 1, '1:i386': 1, '2:x86_64':
> > 1, '2:i386': 1}
> > 2008-05-07 06:10:37,895 [ERROR] koji.build: Session expired
> > 2008-05-07 06:10:37,896 [WARNING] koji.build: Shutting down, please
> > wait...
> >
> >
> > /var/log/kojira.log:
> > 2008-05-07 06:10:25,519 [DEBUG] koji.repo.manager: Reading current repo
> > data
> > 2008-05-07 06:10:25,654 [DEBUG] koji.repo.manager: Repo data: []
> > 2008-05-07 06:10:25,654 [DEBUG] koji.repo.manager: Scanning filesystem
> > for repos
> >
> >
> >
> >
> > --
> > Fedora-buildsys-list mailing list
> > Fedora-buildsys-list@redhat.com
> > https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
>
> --
> Fedora-buildsys-list mailing list
> Fedora-buildsys-list@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
--
Fedora-
Re: koji clone-tag?
On Mon, 2008-04-21 at 20:54 -0400, Doug Chapman wrote: > We are working on building a beta release of our ia64 F9 bits. I have > been digging through the wiki for general release engineering processes > and found this: > > http://fedoraproject.org/wiki/ReleaseEngineering/SOP/FreezingRawhide?highlight=%28ReleaseEngineering/SOP/%29 > > > which shows using the command: > > koji clone-tag dist-f8 f8-final > > however, I find that the clone-tag command does not actually exist. Was > it replaced by something else? We are looking for a way to take the > latest version of all packages and tag them with a tag which will be > used to build the beta release. It does exist, what version of Koji are you using? It requires admin permissions, so it'll only show up in "koji help --admin". For usage, use "koji clone-tag --help". -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji "hidden" packages proposal
On Fri, 2008-03-14 at 19:17 -0600, Stephen John Smoogen wrote: > On Fri, Mar 14, 2008 at 6:55 PM, Dennis Gilmore <[EMAIL PROTECTED]> wrote: > > On Friday 14 March 2008, Mike Bonnet wrote: > > > I've written up a brief proposal about how "hidden" packages may be > > > supported in Koji. The objective of this is to enable building EPEL > > > packages in Koji. I wrote this up fairly quickly, and I'm sure I > > > haven't thought through all the issues, but I wanted to get the ball > > > rolling. Let me know if you have any questions/comments/ideas/issues > > > relating to this proposal. > > > > > > http://people.redhat.com/mikeb/koji/koji-hidden-packages-proposal.html > > > > > > Thanks, > > > Mike > > the tree will have to be nothing like /mnt/koji/packages > > > > instead it will have to be like > > http://download.fedora.redhat.com/pub/fedora/linux/ so that you can use > > rsync and repo mirroring tools to sync the content and keep trees in sync. > > ill leave it up to Seth to explain more but hit micro repository option he > > is > > working on would allow us to pull the existing repodata intothe new > > repodata. > > > > We really only need to have a command that can suck the repodata into the > > database. so we know about the packages. > > > > Another big issue will be that EL is split in so many 'interesting' > ways. You need to be able to either have the build system know of > every seperate channels from RHN for each variation EL-3,4,5 or have > RHN somehow give a conglomerate channel to you of all the packages in > one big bucket. The code from DAG's mrepo sort of does this but would > need some work on getting it to play nicely We're not going to be pulling packages straight from RHN, we'll be manually importing the packages. The proposal allows for multiple alternate directory trees, so it will be possible to have a tree to which we import the RHEL-4 binaries, a tree for RHEL-5 binaries, etc. When a new RHEL update is released, it'll be up to an admin to download the new packages and import them into the proper tree. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji "hidden" packages proposal
On Fri, 2008-03-14 at 19:55 -0500, Dennis Gilmore wrote: > On Friday 14 March 2008, Mike Bonnet wrote: > > I've written up a brief proposal about how "hidden" packages may be > > supported in Koji. The objective of this is to enable building EPEL > > packages in Koji. I wrote this up fairly quickly, and I'm sure I > > haven't thought through all the issues, but I wanted to get the ball > > rolling. Let me know if you have any questions/comments/ideas/issues > > relating to this proposal. > > > > http://people.redhat.com/mikeb/koji/koji-hidden-packages-proposal.html > > > > Thanks, > > Mike > the tree will have to be nothing like /mnt/koji/packages > > instead it will have to be like > http://download.fedora.redhat.com/pub/fedora/linux/ so that you can use > rsync and repo mirroring tools to sync the content and keep trees in sync. > ill leave it up to Seth to explain more but hit micro repository option he is > working on would allow us to pull the existing repodata intothe new repodata. I'm not really sure what you're talking about. *No one* will be mirroring the hidden packages in the case we're talking about (building EPEL in Koji), these will be RHEL binaries, and only available to the builders. > We really only need to have a command that can suck the repodata into the > database. so we know about the packages. We have a command that can suck data about packages in to the database. It's "koji import". The proposal is designed to work within the framework of that we already have in Koji, to try to minimize the number of changes and thus the time it'll take to implement. The micro repository option Seth has talked about sounds interesting, but it doesn't really help with the issue of making packages available to Koji builders without making them available to the world. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Koji "hidden" packages proposal
I've written up a brief proposal about how "hidden" packages may be supported in Koji. The objective of this is to enable building EPEL packages in Koji. I wrote this up fairly quickly, and I'm sure I haven't thought through all the issues, but I wanted to get the ball rolling. Let me know if you have any questions/comments/ideas/issues relating to this proposal. http://people.redhat.com/mikeb/koji/koji-hidden-packages-proposal.html Thanks, Mike -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Thu, 2008-02-28 at 15:21 -0500, Bill Nottingham wrote: > Mike Bonnet ([EMAIL PROTECTED]) said: > > > Actually, do we even know if disabling force tag can be worked around? > > > > Not sure what you're asking here. There's no reason force-tag is > > required for any part of the package maintenance workflow. > > I mean, "even if you remove force-tag from the Makefile, you can still > move the tag yourself". Which I'm fairly sure you can do. We can make tags really immutable by adding a taginfo script to Fedora CVS, and "exit 1" when a "mov" operation (tag -F) is requested. This is essentially the way we prevent the same tag from being applied on multiple branches today. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Thu, 2008-02-28 at 13:45 -0500, Jesse Keating wrote: > On Thu, 28 Feb 2008 18:24:59 + > Paul Howarth <[EMAIL PROTECTED]> wrote: > > > I've used it after I've done a "make tag" whilst forgetting to commit > > changes first, so the tag was applied to the wrong version of the spec > > file etc. Force tagging is useful for correcting this error. > > Couldn't that be solved by making 'make tag' abort if there are > unchecked in files in the dir, particularly .spec ? It already should. "make tag" uses "cvs tag -c". $ cvs -H tag Usage: cvs tag [-bcdFflR] [-r rev|-D date] tag [files...] -c Check that working files are unmodified. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Thu, 2008-02-28 at 11:58 -0500, Bill Nottingham wrote: > Mike Bonnet ([EMAIL PROTECTED]) said: > > If a transient build error (build system hiccup) prevented the build > > from completing, you should be able to rebuild from the same tag. > > > > Yes, I know there are some failure modes where this does not work, but > > we're working on addressing those, and they should be the minority of > > cases. > > True, but if it's something like 'a build dependent package is broken', > you're unlikely to remember to just re-push the task two days later > when it's fixed. I don't see how disabling force-tagging has any effect on this. Remember you can rebuild with the same tag if the build fails. > In any case, turning off force tag would lead to > either: > > - needless release & changelog inflation Do we require people to add a changelog entry for *every* increment of the release number? If you're just bumping the release for a rebuild, it seems reasonable to not bother with the changelog entry. > or > > - usage of scratch builds just to test that it builds everywhere > If the build fails because of a problem with a dependent package, you can rebuild from the same tag once the dependent package is fixed. If the build fails because of a problem with the spec file or source, you need to change something to make it build successfully. Bumping the release and adding a changelog entry seems appropriate in this case. I'd argue that even *now* we should be discouraging force-tag in this case. > Actually, do we even know if disabling force tag can be worked around? Not sure what you're asking here. There's no reason force-tag is required for any part of the package maintenance workflow. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Thu, 2008-02-28 at 11:12 -0500, Bill Nottingham wrote: > Mike Bonnet ([EMAIL PROTECTED]) said: > > I think immutable tags are the answer here. We already kind of assume > > tags don't change once they're built, we might as well enforce it. I > > know force-tag is convenient, but how much harder is it really to bump > > the revision number instead? > > In the case of transient build errors, it's rather annoying. If a transient build error (build system hiccup) prevented the build from completing, you should be able to rebuild from the same tag. Yes, I know there are some failure modes where this does not work, but we're working on addressing those, and they should be the minority of cases. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Wed, 2008-02-27 at 16:15 -0600, Matt Domsch wrote: > On Tue, Feb 26, 2008 at 06:33:12PM -0600, Dennis Gilmore wrote: > > Ive considered the idea of having a web app to make srpms on demand > > for people also. It would most likely be needed there also. But it > > doesnt help with the historical data we dont have. > > I started the 'correspondingsource' project on fedorahosted.org today > exactly for such a webapp. > > We have just over 85k tags for all the packages in CVS; some going > back to 2004, some only to the F7 days. Not sure yet how much history > was lost during the Core import. > > We've talked about "immutable tags" before. While that would be nice, > I'd be fine with having koji add a tag in its own namespace, either > at package checkout pre-build, or on succcessful build, either way is > fine by me. These tags would not be the same tags as 'make tag' > creates, so users can force-tag if they really feel the need, but we > would more than strongly discourage force tagging on the koji > namespace tags. I'm not really comfortable having an automated system (Koji) modifying the SCM. Giving the Koji builders credentials to modify the SCM in a secure way (without making those credentials available to the world) might be tricky. It also dramatically increases the risk in the event that a builder is compromised. Right now all Koji access to the SCM is read-only, and it should probably stay that way. I think immutable tags are the answer here. We already kind of assume tags don't change once they're built, we might as well enforce it. I know force-tag is convenient, but how much harder is it really to bump the revision number instead? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Dazed and confused....
On Wed, 2008-02-06 at 23:02 +, Bryce wrote: > Just playing around with a miniature version of koji at home, building > off my own box (i386/x86_64) using koji-1.2.5 > > Issue 1. The kernel, because my build host is defined as arch i386, > x86_64 when I pass a kernel build in, it passes "--target i386" which > leads to the usual train wreck of no i386 config > how do I pass in i686 specifically for the kernel? You need to set the "extra arches" that a package gets built for, in addition to the arches defined for that tag. Use: koji set-pkg-arches "i686" dist-foo-build kernel In Fedora Koji the i386 build builds a kernel-headers package. If that's not working for you, you can avoid the i386 build altogether by putting "ExcludeArch: i386" into the spec file. > Issue 2. One of the fun items in xen is that when you're building the > 64bit version there is an odd dependency on pulling in a 32bit > glibc-devel by adding /usr/include/gnu/stubs-32.h to the dependency > list, when building in x86_64 that dependancy cannot get satisfied. How > can I beat the tag into realizing it can pull from the i386 repo to > fulfill that requirement? Fedora uses the glibc32 and glibc64 packages to provide these dependencies. A bit of a hack, but it works: http://cvs.fedoraproject.org/viewcvs/rpms/glibc32/devel/ http://cvs.fedoraproject.org/viewcvs/rpms/glibc64/devel/ Koji won't let you pull i386 packages into a x86_64 build environment, or vice versa. The build environments are single-arch only. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
[PATCH] set the current working directory in the chroot
This patch allows you to set the current working directory (in the
chroot) before running a command with --chroot. This avoids the need to
pass shell snippets ('cd /some/path && /run/cmd') to mock when running a
command that expects to executed from a certain directory. It's useful
when using --copyin to setup the environment before running a command.
>From e4071d1d41a62ccf4461dfab958f9325edf30c97 Mon Sep 17 00:00:00 2001
From: Mike Bonnet <[EMAIL PROTECTED]>
Date: Thu, 24 Jan 2008 17:09:06 -0500
Subject: [PATCH] optionally set the current working directory (in the chroot)
before running command with --chroot
---
docs/mock.1 |3 +++
py/mock.py |8 ++--
py/mock/util.py | 13 +
3 files changed, 18 insertions(+), 6 deletions(-)
diff --git a/docs/mock.1 b/docs/mock.1
index 38c3233..531d117 100644
--- a/docs/mock.1
+++ b/docs/mock.1
@@ -140,6 +140,9 @@ Fail build if rpmbuild takes longer than 'timeout' seconds
\fB\-\-unpriv\fR
Drop privileges before running command when using --chroot
.TP
+\fB\-\-cwd=\fR\fIDIR\fP
+Change to the specified directory (relative to the chroot) before running
command when using --chroot
+.TP
\fB\-q\fR, \fB\-\-quiet\fR
Be quiet.
.TP
diff --git a/py/mock.py b/py/mock.py
index f422a33..d5afbbe 100755
--- a/py/mock.py
+++ b/py/mock.py
@@ -152,6 +152,10 @@ def command_parse(config_opts):
" seconds ")
parser.add_option("--unpriv", action="store_true", default=False,
help="Drop privileges before running command when using
--chroot")
+parser.add_option("--cwd", action="store", default=None,
+ metavar="DIR",
+ help="Change to the specified directory (relative to the
chroot)"
+ " before running command when using --chroot")
# verbosity
parser.add_option("-v", "--verbose", action="store_const", const=2,
@@ -536,9 +540,9 @@ def main(ret):
chroot._mountall()
if options.unpriv:
chroot.doChroot(args, shell=shell,
-uid=chroot.chrootuid, gid=chroot.chrootgid)
+uid=chroot.chrootuid, gid=chroot.chrootgid,
cwd=options.cwd)
else:
-chroot.doChroot(args, shell=shell)
+chroot.doChroot(args, shell=shell, cwd=options.cwd)
finally:
chroot._umountall()
diff --git a/py/mock/util.py b/py/mock/util.py
index f93f98b..65cc995 100644
--- a/py/mock/util.py
+++ b/py/mock/util.py
@@ -201,6 +201,10 @@ def condChroot(chrootPath):
os.chroot(chrootPath)
uid.setresuid(saved['ruid'], saved['euid'])
+def condChdir(cwd):
+if cwd is not None:
+os.chdir(cwd)
+
def condDropPrivs(uid, gid):
if gid is not None:
os.setregid(gid, gid)
@@ -245,12 +249,12 @@ def logOutput(fds, logger, returnOutput=1, start=0,
timeout=0):
# The "Not-as-complicated" version
#
decorate(traceLog())
-def do(command, shell=False, chrootPath=None, timeout=0, raiseExc=True,
returnOutput=0, uid=None, gid=None, personality=None, *args, **kargs):
+def do(command, shell=False, chrootPath=None, cwd=None, timeout=0,
raiseExc=True, returnOutput=0, uid=None, gid=None, personality=None, *args,
**kargs):
logger = kargs.get("logger", getLog())
output = ""
start = time.time()
-preexec = ChildPreExec(personality, chrootPath, uid, gid)
+preexec = ChildPreExec(personality, chrootPath, cwd, uid, gid)
try:
child = None
logger.debug("Executing command: %s" % command)
@@ -292,9 +296,10 @@ def do(command, shell=False, chrootPath=None, timeout=0,
raiseExc=True, returnOu
return output
class ChildPreExec(object):
-def __init__(self, personality, chrootPath, uid, gid):
+def __init__(self, personality, chrootPath, cwd, uid, gid):
self.personality = personality
self.chrootPath = chrootPath
+self.cwd = cwd
self.uid = uid
self.gid = gid
@@ -303,4 +308,4 @@ class ChildPreExec(object):
condPersonality(self.personality)
condChroot(self.chrootPath)
condDropPrivs(self.uid, self.gid)
-
+condChdir(self.cwd)
--
1.5.3.3
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] add --unpriv option to drop privileges when running a command with --chroot
On Thu, 2008-01-24 at 16:04 -0500, Mike Bonnet wrote:
> On Thu, 2008-01-24 at 15:42 -0500, Mike Bonnet wrote:
> > This patch adds a --unpriv option that will cause privileges to be
> > dropped before running a command with --chroot. This can be used to
> > more closely simulate the environment used when running rpmbuilds.
>
> Let me try that again...
Ok, the attachments are getting stripped off for some reason, trying
inline...
>From 85e14d38aec32cf20d7f2bbdc77044d41c32a0a2 Mon Sep 17 00:00:00 2001
From: Mike Bonnet <[EMAIL PROTECTED]>
Date: Thu, 24 Jan 2008 15:37:15 -0500
Subject: [PATCH] optionally drop privileges when running a command with --chroot
---
docs/mock.1 |3 +++
py/mock.py |8 +++-
2 files changed, 10 insertions(+), 1 deletions(-)
diff --git a/docs/mock.1 b/docs/mock.1
index beaf2fb..38c3233 100644
--- a/docs/mock.1
+++ b/docs/mock.1
@@ -137,6 +137,9 @@ Change directory where config files are found
\fB\-\-rpmbuild_timeout=\fR\fISECONDS\fP
Fail build if rpmbuild takes longer than 'timeout' seconds
.TP
+\fB\-\-unpriv\fR
+Drop privileges before running command when using --chroot
+.TP
\fB\-q\fR, \fB\-\-quiet\fR
Be quiet.
.TP
diff --git a/py/mock.py b/py/mock.py
index 4a589bc..f422a33 100755
--- a/py/mock.py
+++ b/py/mock.py
@@ -150,6 +150,8 @@ def command_parse(config_opts):
dest="rpmbuild_timeout", type="int", default=None,
help="Fail build if rpmbuild takes longer than 'timeout'"
" seconds ")
+parser.add_option("--unpriv", action="store_true", default=False,
+ help="Drop privileges before running command when using
--chroot")
# verbosity
parser.add_option("-v", "--verbose", action="store_const", const=2,
@@ -532,7 +534,11 @@ def main(ret):
chroot._resetLogging()
try:
chroot._mountall()
-chroot.doChroot(args, shell=shell)
+if options.unpriv:
+chroot.doChroot(args, shell=shell,
+uid=chroot.chrootuid, gid=chroot.chrootgid)
+else:
+chroot.doChroot(args, shell=shell)
finally:
chroot._umountall()
--
1.5.3.3
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] add --unpriv option to drop privileges when running a command with --chroot
On Thu, 2008-01-24 at 15:42 -0500, Mike Bonnet wrote: > This patch adds a --unpriv option that will cause privileges to be > dropped before running a command with --chroot. This can be used to > more closely simulate the environment used when running rpmbuilds. Let me try that again... 0001-optionally-drop-privileges-when-running-a-command-wi.patch Description: application/mbox -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
[PATCH] add --unpriv option to drop privileges when running a command with --chroot
This patch adds a --unpriv option that will cause privileges to be dropped before running a command with --chroot. This can be used to more closely simulate the environment used when running rpmbuilds. 0001-optionally-drop-privileges-when-running-a-command-wi.patch Description: application/mbox -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
[PATCH] do mounts when running mock --chroot
Here's a small patch to handle mounting things into the chroot when
using the --chroot command, the same way we do when using --rebuild.
--- mock.py.orig2008-01-16 13:42:21.0 -0500
+++ mock.py.mounts 2008-01-22 17:10:24.0 -0500
@@ -507,7 +507,12 @@
log.info("Running in chroot: %s" % args)
chroot.tryLockBuildRoot()
chroot._resetLogging()
-chroot.doChroot(args)
+
+try:
+chroot._mountall()
+chroot.doChroot(args)
+finally:
+chroot._umountall()
elif options.mode == 'installdeps':
if len(args) == 0:
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] Create the dev/full device, some packages use it during make check.
On Thu, 2008-01-03 at 09:43 -0500, Jesse Keating wrote: > There are a few packages which use /dev/full to generate nospace left > or other such return values for the use in make check. We should > create that in mock. Here is a simple patch that adds it. Shouldn't that be os.makedev(1, 7)? $ ls -la /dev/full crw-rw-rw- 1 root root 1, 7 2007-12-29 17:34 /dev/full -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mock not processing /etc/profile.d/*, not a login shell?
On Tue, 2007-12-04 at 08:50 -0600, Rex Dieter wrote: > Jesse Keating wrote: > > > On Tue, 04 Dec 2007 07:55:41 -0600 > > Rex Dieter <[EMAIL PROTECTED]> wrote: > > > >> Sigh, so can someone come up with a better solution to "I want to > >> install pkg foo, where it defines env vars, and pkgs consuming/using > >> pkg foo shouldn't need to know it's implementation details (ie, which > >> file in /etc/profile.d/ needs to source'd)" ? > > > > How do you plan to solve this when installing on a real system? User > > installs a bunch of -devel packages, tries to do an rpmbuild, fail. > > What then? > > The easiest (and only) solution I see so far is my initial suggestion: > "start a (new) login shell". > > If that's not acceptable, then we're stuck with the status-quo. Why not convert qt to use pkg-config like so many other projects do, rather than stuffing build configuration details into my environment, where I don't care about them 99% of the time? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RHEL 5.1
On Thu, 2007-11-08 at 16:35 -0500, rob myers wrote:
> I know RHEL is a bit off topic for this list, but bear with me. I
> import all the RHEL5 bits into koji and then use koji to build rpms for
> RHEL5. The release of RHEL 5.1 gave me a few kinks to work out:
>
> The RHEL5 supplementary channel includes some noarch packages which do
> not come with source rpms. This breaks the assumption in mash that
> excludearch and exclusive arch can safely be initialized on SRPMS only.
> I know that mash is not designed for this use case, but I thought it
> might be worth pointing out. The patch I used is below.
>
> diff --git a/mash/__init__.py b/mash/__init__.py
> index b02d6c8..6e7a496 100644
> --- a/mash/__init__.py
> +++ b/mash/__init__.py
> @@ -209,6 +209,18 @@ class Mash:
> # now deal with noarch
> for pkg in noarch:
> for target_arch in self.config.arches:
> +
> +# if excludearch is not set this build likely has no src.rpm
> +# so set excludearch and exclusivearch from the binary
> +if pkg['build_id'] fg
> not in excludearch:
> +path =
> os.path.join(koji.pathinfo.build(builds_hash[pkg['build_id']]),
> koji.pathinfo.rpm(pkg))
> +fn = open(path, 'r')
> +hdr = koji.get_rpm_header(fn)
> +excludearch[pkg['build_id']] = hdr['EXCLUDEARCH']
> +exclusivearch[pkg['build_id']] = hdr['EXCLUSIVEARCH']
> +fn.close()
> +continue
> +
> if (excludearch[pkg['build_id']] and
> has_any(masharch.compat[target_arch], excludearch[pkg['build_id']])) or \
> (exclusivearch[pkg['build_id']] and not
> has_any(masharch.compat[target_arch], exclusivearch[pkg['build_id']])):
> print "Excluding %s.%s from %s due to
> EXCLUDEARCH/EXCLUSIVEARCH" % (pkg['name'], pkg['arch'], target_arch)
>
>
> Another problem was that I had already imported RHEL 5.1 beta, which has
> the same rpms in RHEL 5.1 signed with a different key- one that I do not
> have access to. Since I wanted the rpms written out with the release
> key, I added a function take the 5.1 release sighdrs and add them to the
> previously imported RHEL 5.1 beta rpms. Would anyone else ever need this
> functionality? The patch I used is below.
>
> diff --git a/hub/kojihub.py b/hub/kojihub.py
> index f9ba39a..c5a1261 100644
> --- a/hub/kojihub.py
> +++ b/hub/kojihub.py
> @@ -4706,6 +4706,15 @@ class RootExports(object):
> context.session.assertPerm('sign')
> return add_rpm_sig(an_rpm, base64.decodestring(data))
>
> +def addRPMSigByPath(self, an_rpm, path):
> +"""Store a signature header for an rpm
> +
> +an_rpm: N-V-R.a (for example)
> +path: the path to the signed rpm
> +"""
> +context.session.assertPerm('sign')
> +return add_rpm_sig(an_rpm, koji.rip_rpm_sighdr(path))
> +
> findBuildID = staticmethod(find_build_id)
> getTagID = staticmethod(get_tag_id)
> getTag = staticmethod(get_tag)
>
>
> Comments?
Does the "koji import-sig" command not do what you want?
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RFC - Patch - koji SCM generalization
On Tue, 2007-11-06 at 11:16 -0500, rob myers wrote:
> On Mon, 2007-11-05 at 18:56 -0500, Mike Bonnet wrote:
> >
> > The patch has been committed to Koji:
> >
> > http://git.fedoraproject.org/?p=hosted/koji;a=commit;h=27ac942683d8bbfd28c3b6bbc8475b4cbada0c2c
> >
> > I made some pretty significant changes to make the SCM class more
> > object-y, remove some code duplication, make it more configurable at
> > runtime, etc., but the framework setup in the original patch is there.
> > I've tested it fairly completely and everything seems to be working as
> > expected. Note that when using the "+ssh://" access methods, you're
> > responsible for getting the required login credentials (host keys,
> > private keys, Kerberos tickets) into the right place on the builders
> > yourself.
>
> mike-
>
> your patch is much tidier than mine, thanks for fixing it up! i really
> like the removal of scmtype from the configuration file. i'm mixed on
> the removal of scmusername because i'm not sure that koji clients should
> specify the username used by the koji builder to perform ssh checkouts.
> however, while i thought it worth pointing out, it does not seem too big
> a deal to disclose this username.
>
> i did some testing with my svn+ssh repository and found the patch below
> necessary- it creates ../common only if it does not already exist.
>
> diff --git a/builder/kojid b/builder/kojid
> index b436a19..5c5d665 100755
> --- a/builder/kojid
> +++ b/builder/kojid
> @@ -2506,7 +2507,8 @@ class SCM(object):
> (self.scmtype, ' '.join(update_checkout_cmd),
> os.path.basename(logfile))
>
> # Required by Dist CVS layout
> -os.symlink('%s/common' % scmdir, '%s/../common' % sourcedir)
> +if not os.path.exists('%s/../common' % sourcedir):
> +os.symlink('%s/common' % scmdir, '%s/../common' %
> sourcedir)
>
> def get_options():
> """process options from command line and config file"""
>
> thanks again for adding this functionality to koji. :)
The patch looks good, I've applied it. Just out of curiousity, where
did the common/ directory come from, an svn:external?
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RFC - Patch - koji SCM generalization
On Tue, 2007-10-23 at 15:14 -0400, rob myers wrote: > On Tue, 2007-10-23 at 14:30 -0400, Mike Bonnet wrote: > > On Tue, 2007-10-23 at 14:13 -0400, rob myers wrote: > > > attached is a patch that attempts to generalize checking out the > > > components used to build an SRPM. this patch supports CVS, GIT, and > > > SVN, but only CVS and SVN have been tested. the idea is to provide the > > > infrastructure for different SCM systems to be configured at run-time so > > > that users can choose their favorite system. > > > > It would also be nice if rather than assuming the existence of the > > "common/" module and hard-coding the "make srpm" command, there was some > > way to specify what modules were necessary to checkout, and what command > > to run to create a srpm. This should probably be configurable > > per-repository. It might make sense to add a new database table to > > store this information in a more flexible way than possible with the > > config files. > > the existing design is (currently) sufficient for my needs. what you > describe is more flexible and is probably a better design, but is it > needed? > > > However, this is a great start. I'm going to review the patch more > > carefully, and I'll see about merging it soon (after possibly removing > > the scmtype config option I mentioned earlier). > > thanks for your feedback, and remove cruft as necessary. :) The patch has been committed to Koji: http://git.fedoraproject.org/?p=hosted/koji;a=commit;h=27ac942683d8bbfd28c3b6bbc8475b4cbada0c2c I made some pretty significant changes to make the SCM class more object-y, remove some code duplication, make it more configurable at runtime, etc., but the framework setup in the original patch is there. I've tested it fairly completely and everything seems to be working as expected. Note that when using the "+ssh://" access methods, you're responsible for getting the required login credentials (host keys, private keys, Kerberos tickets) into the right place on the builders yourself. Thanks for the patch Rob, and if you have any other suggestions or improvements, please let me know. Keep those patches coming! :) -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RFC - Patch - koji SCM generalization
On Tue, 2007-10-23 at 14:13 -0400, rob myers wrote: > attached is a patch that attempts to generalize checking out the > components used to build an SRPM. this patch supports CVS, GIT, and > SVN, but only CVS and SVN have been tested. the idea is to provide the > infrastructure for different SCM systems to be configured at run-time so > that users can choose their favorite system. It would also be nice if rather than assuming the existence of the "common/" module and hard-coding the "make srpm" command, there was some way to specify what modules were necessary to checkout, and what command to run to create a srpm. This should probably be configurable per-repository. It might make sense to add a new database table to store this information in a more flexible way than possible with the config files. However, this is a great start. I'm going to review the patch more carefully, and I'll see about merging it soon (after possibly removing the scmtype config option I mentioned earlier). Thanks a lot! -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RFC - Patch - koji SCM generalization
On Tue, 2007-10-23 at 14:13 -0400, rob myers wrote: > attached is a patch that attempts to generalize checking out the > components used to build an SRPM. this patch supports CVS, GIT, and > SVN, but only CVS and SVN have been tested. the idea is to provide the > infrastructure for different SCM systems to be configured at run-time so > that users can choose their favorite system. > > is there a better approach? did i miss something obvious? general > comments? It looks pretty good, but I wonder if you need to make scmtype a config option? I'd rather have Koji support all scmtypes all the time and have some other method for restricting what URLs are valid locations to download the source from. Maybe a list of valid hostnames or hostname/path pairs? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mock improvements
On Tue, 2007-09-25 at 14:11 -0500, Michael E Brown wrote: > So here is the list of things that have been requested lately and I'll > be working on a few of these over the next few weeks. If anybody has any > input, I'd take it. As I start on each, I'll most likely email the > mailing list with the outline of what I'm doing. > > If anybody has existing patches for these (against current mock git), > all the better... :) > > 1) more reliable mount/umount > several people have pointed out instances where mock exits leaving > mounts behind (specifically /dev), and the next invokation of mock > ends up 'rm -rf' the host machine's /dev. Bad > > 2) caching yum downloads > several people have commented that the autocache stuff is great for > speeding up builds, others say that it can sometimes be bad for > reproducability, and that simply saving the yum cache dir would be > better. > > 3) ccache integration > This is a new one that I havent seen before, but should significantly > speed up builds for people who often do > rebuild-the-entire-distribution-type things. I'm told by some that > this is bad for reproducability, but good for speeding up builds when > you are just sanity checking or when that small reproducability hit > doesnt matter. I've also seen lots of empirical data that ccache > should not cause any problems. This will be have to be specifically > enabled through a commandline or config file option, so those who care > can turn it on/off. > > 4) distcc integration > Pretty much the same case as ccache. Has more things that need thought > than the ccache case, above, though. A method for cleaning up stale/orphaned processes that get created during a build was proposed a while ago: https://bugzilla.redhat.com/show_bug.cgi?id=221351 I'll leave it to you as to whether or not this is the correct implementation, but I certainly think it's a good idea. I see rogue processes consuming resources on the build machines all the time. The process-cleanup code should probably be run before mock exits, whether the build completed successfully or not. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Security issue: Can't build mediawiki - F7 thinks it's F8
On Mon, 2007-08-06 at 15:00 +0200, Axel Thimm wrote: > On Mon, Aug 06, 2007 at 07:51:00AM -0500, Dennis Gilmore wrote: > > Once upon a time Monday 06 August 2007, Jesse Keating wrote: > > > On Mon, 6 Aug 2007 14:18:36 +0200 > > > > > > Axel Thimm <[EMAIL PROTECTED]> wrote: > > > > Typo? > > > > > > Not exactly. Expression mismatch. The tag was /applied/ on the devel/ > > > branch, so when cvs is asked for that tag, it tries to pull it from > > > devel/ and bad things happen. > > > > > > > > in devel, koji gets a little confused when making the srpm for you. > > > > > You'll most likely need to bump/tag on F-7 then you can build and it > > > > > will get the proper .fc7 tag to it. > > > > > > > > Hm, this sounds more like a bug that should be fixed in koji. Wouldn't > > > > that apply to any kind of branching CVS, e.g. koji inherits bad tags > > > > to branches? This bug only surfaces if the tagsing and building is > > > > intermitted by the branch, but consider adding new archs to Fedora, it > > > > will hit all packages (as the build for the new arch will be after the > > > > branching, unless new archs are limited to devel). > > > > > > I'm not entirely sure how this is going to be handled. It probably > > > does need looking into, something deep in the cvs "branching" scripts > > > we use. My cvs-fu isn't nearly that strong :( > > > > probably need to call make tag after creating the branch. > > The problem was that make tag has been created before the branch, but > the make build afterwards. It is also not possible to rerun make tag. The problem here is actually that the tag created on devel/ didn't include a "branch" file (that file doesn't get created until the branch-creation scripts are run). In the absence of a "branch" file, Makefile.common assumes you're on devel/ and expands the %dist tag to the values defined for devel in common/branches (currently .fc8). Basically, tagging before the branch point and building after it is not supported. After the branch point, any builds need to bump the revision and run "make tag" before they can build. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RFC: utility of 'orphansbuild' patch to mock-helper (BZ#221351)?
On Tue, 2007-07-10 at 11:53 -0500, Clark Williams wrote: > Jan Kratochvil has submitted a patch to mock that adds the 'orphanskill' > command to > mock-helper (a setuid root program used by mock). The patch traverses the > /proc > directory, looking for tasks with a "root" link that matches the chroot > currently in > use, and sends a SIGKILL to each matching task. > > As far as I can tell this is only useful to the GDB build. The testsuite for > GDB > seems to have some either abnormal terminations or so other oddity that > leaves jobs > hanging. I've looked at the C code and it looks well written, without obvious > security holes. > > I've mixed feelings regarding adding the command. Michael and I have been > fairly > resistant to adding things to mock-helper, on the general principle that > adding > features to a setuid root program is fraught with peril. I see the utility of > the > code, but I'm torn as to whether the 'orphanskill' command is sufficiently > useful to > the general community. > > So, that's the question. Is 'orphanskill' worth adding to mock? GDB is not the only build that leaves orphaned processes lying around. I've seen similar behavior when building gcc, glibc, and mysql, to name a few. The problems are usually caused by test suites called during the build process, and leaving them around after a build has completed (or failed) can tie up system resources or in some cases cause subsequent builds to fail. Just as mock cleans up the filesystem after a build, it should probably be cleaning up the process list as well. I'd be in favor of adding this patch. Koji could certainly make use of it. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH koji] Set proper permissions (0666) when creating files
On Fri, 2007-05-11 at 18:48 +0200, Enrico Scholz wrote:
> Currently, os.open() is used without specifying the third argument.
> Hence, all files will be created with 0777 perms (minus umask).
>
> Because only datafiles will be uploaded, the exec-bit should be removed.
Indeed it should. Patch applied, thanks a lot!
> Signed-off-by: Enrico Scholz <[EMAIL PROTECTED]>
> ---
>
> builder/kojid |2 +-
> hub/kojihub.py |2 +-
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/builder/kojid b/builder/kojid
> index 6243edd..9a82c60 100755
> --- a/builder/kojid
> +++ b/builder/kojid
> @@ -109,7 +109,7 @@ def log_output(path, args, outfile, uploadpath, cwd=None,
> logerror=0, append=0,
> flags = os.O_CREAT | os.O_WRONLY
> if append:
> flags |= os.O_APPEND
> -fd = os.open(outfile, flags)
> +fd = os.open(outfile, flags, 0666)
> os.dup2(fd, 1)
> if logerror:
> os.dup2(fd, 2)
> diff --git a/hub/kojihub.py b/hub/kojihub.py
> index a3bbe5f..03cb76f 100644
> --- a/hub/kojihub.py
> +++ b/hub/kojihub.py
> @@ -3958,7 +3958,7 @@ class RootExports(object):
> # elif offset == 0:
> # #first chunk, so file should not exist yet
> # raise koji.GenericError, "file already exists: %s" % fn
> -fd = os.open(fn, os.O_RDWR | os.O_CREAT)
> +fd = os.open(fn, os.O_RDWR | os.O_CREAT, 0666)
> # log_error("fd=%r" %fd)
> try:
> if offset == 0 or (offset == -1 and size == len(contents)):
>
> --
> Fedora-buildsys-list mailing list
> Fedora-buildsys-list@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] kojid - BuildNotificationTask traceback
On Sun, 2007-04-22 at 13:22 +0200, Michael Schwendt wrote: > --- kojid~ 2007-04-09 19:23:38.0 +0200 > +++ kojid 2007-04-22 13:18:09.0 +0200 > @@ -2118,7 +2118,7 @@ > if changelog: > changelog = "Changelog:\r\n%s" % changelog > > -from_addr = self.from_addr > +from_addr = options.from_addr > to_addrs = ', '.join(recipients) > subject = self.subject_templ % locals() > message = self.message_templ % locals() Thanks, patch applied. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
