Re: debian-ports.org getting relatively unstable (hppa)
Hi Thorsten,
thanks for your help!
On 12/15/2013 02:59 PM, Thorsten Glaser wrote:
> Helge Deller dixit:
>
>> We noticed, that when we manually binmnu-upload packages, which are
>> already in the *same version* on debian-ports, then debian-ports ACCEPT
>
> When you binNMU packages you add a +b1, +b2, … suffix to their
> versions. ITYM porter upload?
Yes, we did correct binNMU uploads for packages which already existed
in the same version in the repo. But there were lots of packages which
were outdated (the hppa build servers stopped in 2011!) and for those
we just rebuilt from current source and uploaded with the current
version.
>> those packages, but if we then try to apt-get-update those later on, this
>> leads
>> to a "size mismatch" error. I do have the feeling, that this is a
>> problem on debian-ports.
>
> I noticed this problem too, when I accidentally built a package
> I already had uploaded (and totally forgotten about): basically,
> the new *.deb files are accepted but the Packages file still
> contains the checksums etc. from the old *.deb file.
Ok, so it's a generic problem.
> Only way to fix this is to reupload the old *.changes file, or
> to do a binNMU proper. Or to build a newer version, ofc…
Yes, this is how we solved it too (binNMU) then.
>> So, I'm anxious, that if I start the buildd, it will happily build and
>> upload packages
>> which we already uploaded to debian-ports. If this happens we will get more
>> size-mismatch errors.
>
> That’s what you have wanna-build for. Basically, stop doing
> manual uploads without wanna-build locking at least six hours
> before turning on the first buildd. After that time, when you
> want/need to build a package manually, lock it in wanna-build:
> either “take” it for building, or mark as N-F-U.
Ok.
> See here for more info on that:
>
> • https://wiki.debian.org/M68k/Porting#binNMU_notes
> • http://lists.debian.org/debian-68k/2012/12/msg00124.html
> • http://lists.debian.org/debian-68k/2013/10/msg00021.html
Good links. Thanks!
> If you have packages that buildds should never build, for example
> like we had gcc-4.{6,8} for some time, mark them as Not-For-Us;
> otherwise, just take them for building.
>
>> deller@leda:~$ wb info hello . hppa
>
> This the same as “wanna-build -A hppa -d unstable --info hello”.
>
>> But on http://ftp.debian-ports.org/debian/pool-hppa/main/h/hello/ you can
>> see, that the hello-package is already uploaded at version 2.8-4
>
> Indeed. This is bad, new, another / a different problem, and we
> didn’t have this on m68k. (Note that uploads usually take a bit
> until they show up on w-b, hence the need for locking.)
It seems the wb-database was turned off because we didn't had buildd
servers for quite some time. Aurelien will turn it back on again.
>> So, if my buildd now uploads the newly created hello package, I'm sure
>> we will run again into the size-mismatch problem.
>
> Yes, you will definitely run into that problem when you upload
> hello_2.8-4_hppa again.
>
>> My question here on the list would be, if you (other arch-porters) do have
>> an idea
>> on how I should proceed.
>
> Either…
>
>> Best solution would probably be, if the wanna-build database rescans what's
>> in
>> the archive already. Is this possible?
>
> … that (no idea if it’s possible), or make two lists: a list of what
> is currently in the archive for hppa, and a list of packages in the
> Needs-Build or BD-Uninstallable¹ state. Then, for every package in
> the same versions (except +b* sufficēs) in *both* lists, schedule a
> binNMU (e.g. to get hello_2.8-4+b1_hppa). Do note whether it already
> got a binNMU suffix: e.g. aclock.app_0.2.3-3+b4 would need to be
> scheduled for --binNMU=5 to be larger.
>
> You might be able to cheat, e.g. take hello for building, then tell
> it that you uploaded it. But I don’t know why w-b doesn’t register
> that it’s there in the first place, so a rescan, if possible, should
> happen first.
Before Aurelien's answer I was thinking if this could work on leda too:
touch -d2013-01-01 ~/ref
cd /srv/mini-dak/ftp/debian/pool-hppa/main
find . -newer ~/ref | grep .changes$
Basically it would just try to find all packages (.changes) which
we uploaded after january 2012. Then in the next step maybe use the
"--pretend-avail" option of wb to tell it that this package is already
up-to-date. Not sure if this would work though...
But I will now first wait until the wb-database will gets activated
again and check then.
> Hm, only 12 packages here:
> tg@leda:~$ wanna-build -A hppa -d unstable --list=needs-build | less
> But this has more (9043):
> tg@leda:~$ wanna-build -A hppa -d unstable --list=bd-uninstallable | less
>
> ① You need to include BD-Uninstallable because they will happily
> convert to Needs-Build once you upload e.g. perl.
>
>> Or, should I just start the buildd and see what's happening? If we then get
>
> No, get the w-b list consistent first.
Ok.
> According to
> http
Re: debian-ports.org getting relatively unstable (hppa)
Hello Aurelien, On 12/15/2013 09:03 PM, Aurelien Jarno wrote: > On Sun, Dec 15, 2013 at 11:54:43AM +0100, Helge Deller wrote: >> On 12/15/2013 06:32 AM, Dave Land wrote: >>> Not sure what's up at debian-ports.org, but I've been trying to >>> debootstrap 2 different HPPA machines for the last couple days and >>> have been getting a variety of errors (size mismatches, files not >>> found when they were there 20 minutes before, etc. etc.) Somebody may >>> want to look into this before it gets out of hand. Thanks! :) >> >> I maybe should add some more background here, and maybe someone >> here on the list might have an idea on how to proceed. >> >> Background is, that Dave and myself have binmnu-uploaded the necessary >> packages for hppa so that debootstrap worked. Then we proceeded with the >> necessary >> packages for sbuild and schroot, so that I now have a buildd installed >> which should be able to start building packages. I haven't turned it >> on yet though for the reasons which I explain in a few seconds... >> >> In the meantime we have of course uploaded a few more packages which >> now currently break debootstrap. This is fixable manually, but I instead >> of uploading packages manually now, I would prefer to get the buildd >> going instead... So, Dave Land, please wait a little bit... >> >> Now to the reasons why I didn't turned on the buildd yet: >> We noticed, that when we manually binmnu-upload packages, which are >> already in the *same version* on debian-ports, then debian-ports ACCEPT >> those packages, but if we then try to apt-get-update those later on, this >> leads >> to a "size mismatch" error. I do have the feeling, that this is a >> problem on debian-ports. I noticed for example that reprepro usually >> doesn't accept packages of the same version which doesn't seem to be >> the case on debian-ports. > > This is indeed the case, apt-fptarchive keep the checksums corresponding > to first package. That said it hasn't really caused any problem so far. > >> So, I'm anxious, that if I start the buildd, it will happily build and >> upload packages >> which we already uploaded to debian-ports. If this happens we will get more >> size-mismatch errors. > > Well if you leave the build daemons handling the uploads, they will not > build and upload the same package again, and the problem won't happen. Ok, so we should enable the buildd as soon as possble. >> A trivial example: >> On machine buildd.debian-ports.org I run: >> deller@leda:~$ wb info hello . hppa >> * hello/hppa >> | hello: >> | Package : hello >> | Version : 2.8-4 >> | State : Needs-Build >> | Section : devel >> | Priority: source >> | Previous-State : >> | State-Change: 2013-02-18 00:03:36.782007 >> | CalculatedPri : 52 >> | component : main >> | Distribution: sid >> | Notes : out-of-date >> | State-Days : 300 >> | State-Time : 25958430 >> >> So, the package "hello" would need a rebuild according to the wanna-build >> database, >> and that would wb probably tell my buildd who then would start >> building/uploading it. >> But on http://ftp.debian-ports.org/debian/pool-hppa/main/h/hello/ you can >> see, that the hello-package is already uploaded at version 2.8-4 >> So, if my buildd now uploads the newly created hello package, I'm sure >> we will run again into the size-mismatch problem. > > The wanna-build database is not up to date on hppa. I have disabled it > to save some very precious cpu cycles given there are no buildds on hppa > yet. Can you the please start it now again? This would help me to see what's missing. >> Now, Aurelien mentioned last week to me, that this size-mismatch error >> might be because of the "apt-ftparchive cache might have been corrupted for >> hppa". >> I'm not 100% sure about that. > > Ok I wasn't aware the same package have been uploaded multiple time, so > the corruption comes clearly from there. > >> My question here on the list would be, if you (other arch-porters) do have >> an idea >> on how I should proceed. > > I would say stop doing manual upload and start the build daemons. Will do. That was even the plan - just upload enough that debootstrap/sbuild/schroot is installable which it is now. >> Best solution would probably be, if the wanna-build database rescans what's >> in >> the archive already. Is this possible? > > Yes, I can re-enable the hppa wanna-build database if it is actually > useful. Yes, please, turn it on now. I will send you the hppa buildd gpg/ssh details (as mentioned in your other mail to me) in a few moments. >> Or, should I just start the buildd and see what's happening? If we then get >> the size-mismatch errors there is lot of manual work to fix it (unless >> resetting the >> apt-ftparchive on debian-ports would solve this). > > We can rebuild the apt-f
Re: debian-ports.org getting relatively unstable (hppa)
Hi, On Sun, Dec 15, 2013 at 11:54:43AM +0100, Helge Deller wrote: > On 12/15/2013 06:32 AM, Dave Land wrote: > > Not sure what's up at debian-ports.org, but I've been trying to > > debootstrap 2 different HPPA machines for the last couple days and > > have been getting a variety of errors (size mismatches, files not > > found when they were there 20 minutes before, etc. etc.) Somebody may > > want to look into this before it gets out of hand. Thanks! :) > > I maybe should add some more background here, and maybe someone > here on the list might have an idea on how to proceed. > > Background is, that Dave and myself have binmnu-uploaded the necessary > packages for hppa so that debootstrap worked. Then we proceeded with the > necessary > packages for sbuild and schroot, so that I now have a buildd installed > which should be able to start building packages. I haven't turned it > on yet though for the reasons which I explain in a few seconds... > > In the meantime we have of course uploaded a few more packages which > now currently break debootstrap. This is fixable manually, but I instead > of uploading packages manually now, I would prefer to get the buildd > going instead... So, Dave Land, please wait a little bit... > > Now to the reasons why I didn't turned on the buildd yet: > We noticed, that when we manually binmnu-upload packages, which are > already in the *same version* on debian-ports, then debian-ports ACCEPT > those packages, but if we then try to apt-get-update those later on, this > leads > to a "size mismatch" error. I do have the feeling, that this is a > problem on debian-ports. I noticed for example that reprepro usually > doesn't accept packages of the same version which doesn't seem to be > the case on debian-ports. This is indeed the case, apt-fptarchive keep the checksums corresponding to first package. That said it hasn't really caused any problem so far. > So, I'm anxious, that if I start the buildd, it will happily build and upload > packages > which we already uploaded to debian-ports. If this happens we will get more > size-mismatch errors. Well if you leave the build daemons handling the uploads, they will not build and upload the same package again, and the problem won't happen. > A trivial example: > On machine buildd.debian-ports.org I run: > deller@leda:~$ wb info hello . hppa > * hello/hppa > | hello: > | Package : hello > | Version : 2.8-4 > | State : Needs-Build > | Section : devel > | Priority: source > | Previous-State : > | State-Change: 2013-02-18 00:03:36.782007 > | CalculatedPri : 52 > | component : main > | Distribution: sid > | Notes : out-of-date > | State-Days : 300 > | State-Time : 25958430 > > So, the package "hello" would need a rebuild according to the wanna-build > database, > and that would wb probably tell my buildd who then would start > building/uploading it. > But on http://ftp.debian-ports.org/debian/pool-hppa/main/h/hello/ you can > see, that the hello-package is already uploaded at version 2.8-4 > So, if my buildd now uploads the newly created hello package, I'm sure > we will run again into the size-mismatch problem. The wanna-build database is not up to date on hppa. I have disabled it to save some very precious cpu cycles given there are no buildds on hppa yet. > Now, Aurelien mentioned last week to me, that this size-mismatch error > might be because of the "apt-ftparchive cache might have been corrupted for > hppa". > I'm not 100% sure about that. Ok I wasn't aware the same package have been uploaded multiple time, so the corruption comes clearly from there. > My question here on the list would be, if you (other arch-porters) do have an > idea > on how I should proceed. I would say stop doing manual upload and start the build daemons. > Best solution would probably be, if the wanna-build database rescans what's in > the archive already. Is this possible? Yes, I can re-enable the hppa wanna-build database if it is actually useful. > Or, should I just start the buildd and see what's happening? If we then get > the size-mismatch errors there is lot of manual work to fix it (unless > resetting the > apt-ftparchive on debian-ports would solve this). We can rebuild the apt-ftparchive database at some point. Aurelien -- Aurelien Jarno GPG: 1024D/F1BCDB73 aurel...@aurel32.net http://www.aurel32.net -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20131215200337.ga2...@hall.aurel32.net
Re: debian-ports.org getting relatively unstable (hppa)
On 12/15/13 3:54 AM, Helge Deller wrote: On 12/15/2013 06:32 AM, Dave Land wrote: Not sure what's up at debian-ports.org, but I've been trying to debootstrap 2 different HPPA machines for the last couple days and have been getting a variety of errors (size mismatches, files not found when they were there 20 minutes before, etc. etc.) Somebody may want to look into this before it gets out of hand. Thanks! :) I maybe should add some more background here, and maybe someone here on the list might have an idea on how to proceed. Background is, that Dave and myself have binmnu-uploaded the necessary packages for hppa so that debootstrap worked. Then we proceeded with the necessary packages for sbuild and schroot, so that I now have a buildd installed which should be able to start building packages. I haven't turned it on yet though for the reasons which I explain in a few seconds... In the meantime we have of course uploaded a few more packages which now currently break debootstrap. This is fixable manually, but I instead of uploading packages manually now, I would prefer to get the buildd going instead... So, Dave Land, please wait a little bit... Now to the reasons why I didn't turned on the buildd yet: We noticed, that when we manually binmnu-upload packages, which are already in the *same version* on debian-ports, then debian-ports ACCEPT those packages, but if we then try to apt-get-update those later on, this leads to a "size mismatch" error. I do have the feeling, that this is a problem on debian-ports. I noticed for example that reprepro usually doesn't accept packages of the same version which doesn't seem to be the case on debian-ports. So, I'm anxious, that if I start the buildd, it will happily build and upload packages which we already uploaded to debian-ports. If this happens we will get more size-mismatch errors. A trivial example: On machine buildd.debian-ports.org I run: deller@leda:~$ wb info hello . hppa * hello/hppa | hello: | Package : hello | Version : 2.8-4 | State : Needs-Build | Section : devel | Priority: source | Previous-State : | State-Change: 2013-02-18 00:03:36.782007 | CalculatedPri : 52 | component : main | Distribution: sid | Notes : out-of-date | State-Days : 300 | State-Time : 25958430 So, the package "hello" would need a rebuild according to the wanna-build database, and that would wb probably tell my buildd who then would start building/uploading it. But on http://ftp.debian-ports.org/debian/pool-hppa/main/h/hello/ you can see, that the hello-package is already uploaded at version 2.8-4 So, if my buildd now uploads the newly created hello package, I'm sure we will run again into the size-mismatch problem. Now, Aurelien mentioned last week to me, that this size-mismatch error might be because of the "apt-ftparchive cache might have been corrupted for hppa". I'm not 100% sure about that. My question here on the list would be, if you (other arch-porters) do have an idea on how I should proceed. Best solution would probably be, if the wanna-build database rescans what's in the archive already. Is this possible? Or, should I just start the buildd and see what's happening? If we then get the size-mismatch errors there is lot of manual work to fix it (unless resetting the apt-ftparchive on debian-ports would solve this). Any other ideas? Helge Thanks Helge, I'll just put the A500 on standby for a while until things stabilize a bit. I saw Thorsten Glaser's comments over in the debian-ports list, and although it doesn't make a whole lot of sense to me, since I don't fully understand how packages get processed at the repository when the buildd server uploads (or somebody manually uploads). It seems odd that debian-ports would accept a package with the same name/version number... unless it's looking at the md5 signature instead of the filename?? Is that even possible? I would think those would be unique for each file uploaded. Might be a solution to rework things where the server would look at the filename *and* the md5 sig to determine if the filename would need to be auto-incremented to the next iteration/update (e.g. hello-2.1.1-1 to hello-2.1.1-2) Just a thought... I'll check back later. :) Dave L. -- -- Dave Land Land Computer Service xmecha...@landcomp.net ICQ: 676030523 -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52adda03.6030...@landcomp.net
Re: debian-ports.org getting relatively unstable (hppa)
Helge Deller dixit:
>We noticed, that when we manually binmnu-upload packages, which are
>already in the *same version* on debian-ports, then debian-ports ACCEPT
When you binNMU packages you add a +b1, +b2, … suffix to their
versions. ITYM porter upload?
>those packages, but if we then try to apt-get-update those later on, this leads
>to a "size mismatch" error. I do have the feeling, that this is a
>problem on debian-ports.
I noticed this problem too, when I accidentally built a package
I already had uploaded (and totally forgotten about): basically,
the new *.deb files are accepted but the Packages file still
contains the checksums etc. from the old *.deb file.
Only way to fix this is to reupload the old *.changes file, or
to do a binNMU proper. Or to build a newer version, ofc…
>So, I'm anxious, that if I start the buildd, it will happily build and upload
>packages
>which we already uploaded to debian-ports. If this happens we will get more
>size-mismatch errors.
That’s what you have wanna-build for. Basically, stop doing
manual uploads without wanna-build locking at least six hours
before turning on the first buildd. After that time, when you
want/need to build a package manually, lock it in wanna-build:
either “take” it for building, or mark as N-F-U.
See here for more info on that:
• https://wiki.debian.org/M68k/Porting#binNMU_notes
• http://lists.debian.org/debian-68k/2012/12/msg00124.html
• http://lists.debian.org/debian-68k/2013/10/msg00021.html
If you have packages that buildds should never build, for example
like we had gcc-4.{6,8} for some time, mark them as Not-For-Us;
otherwise, just take them for building.
>deller@leda:~$ wb info hello . hppa
This the same as “wanna-build -A hppa -d unstable --info hello”.
>But on http://ftp.debian-ports.org/debian/pool-hppa/main/h/hello/ you can
>see, that the hello-package is already uploaded at version 2.8-4
Indeed. This is bad, new, another / a different problem, and we
didn’t have this on m68k. (Note that uploads usually take a bit
until they show up on w-b, hence the need for locking.)
>So, if my buildd now uploads the newly created hello package, I'm sure
>we will run again into the size-mismatch problem.
Yes, you will definitely run into that problem when you upload
hello_2.8-4_hppa again.
>My question here on the list would be, if you (other arch-porters) do have an
>idea
>on how I should proceed.
Either…
>Best solution would probably be, if the wanna-build database rescans what's in
>the archive already. Is this possible?
… that (no idea if it’s possible), or make two lists: a list of what
is currently in the archive for hppa, and a list of packages in the
Needs-Build or BD-Uninstallable¹ state. Then, for every package in
the same versions (except +b* sufficēs) in *both* lists, schedule a
binNMU (e.g. to get hello_2.8-4+b1_hppa). Do note whether it already
got a binNMU suffix: e.g. aclock.app_0.2.3-3+b4 would need to be
scheduled for --binNMU=5 to be larger.
You might be able to cheat, e.g. take hello for building, then tell
it that you uploaded it. But I don’t know why w-b doesn’t register
that it’s there in the first place, so a rescan, if possible, should
happen first.
Hm, only 12 packages here:
tg@leda:~$ wanna-build -A hppa -d unstable --list=needs-build | less
But this has more (9043):
tg@leda:~$ wanna-build -A hppa -d unstable --list=bd-uninstallable | less
① You need to include BD-Uninstallable because they will happily
convert to Needs-Build once you upload e.g. perl.
>Or, should I just start the buildd and see what's happening? If we then get
No, get the w-b list consistent first.
According to
http://ftp.debian-ports.org/debian/dists/unstable/main/binary-hppa/Packages.bz2
hello is at version 2.8-4 just fine… hmm. So the apt-ftparchive database
seems to be correct.
This is all quite complicated, so feel free to ask around when we
can help you out again, no need for every arch to go through all
of this by themselves, figure out best practices again, etc.
HTH & HAND,
//mirabilos
--
If Harry Potter gets a splitting headache in his scar
when he’s near Tom Riddle (aka Voldemort),
does Tom get pain in the arse when Harry is near him?
-- me, wondering why it’s not Jerry Potter………
--
To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive:
http://lists.debian.org/pine.bsm.4.64l.1312151335210.21...@herc.mirbsd.org
Re: debian-ports.org getting relatively unstable (hppa)
On 12/15/2013 06:32 AM, Dave Land wrote: > Not sure what's up at debian-ports.org, but I've been trying to > debootstrap 2 different HPPA machines for the last couple days and > have been getting a variety of errors (size mismatches, files not > found when they were there 20 minutes before, etc. etc.) Somebody may > want to look into this before it gets out of hand. Thanks! :) I maybe should add some more background here, and maybe someone here on the list might have an idea on how to proceed. Background is, that Dave and myself have binmnu-uploaded the necessary packages for hppa so that debootstrap worked. Then we proceeded with the necessary packages for sbuild and schroot, so that I now have a buildd installed which should be able to start building packages. I haven't turned it on yet though for the reasons which I explain in a few seconds... In the meantime we have of course uploaded a few more packages which now currently break debootstrap. This is fixable manually, but I instead of uploading packages manually now, I would prefer to get the buildd going instead... So, Dave Land, please wait a little bit... Now to the reasons why I didn't turned on the buildd yet: We noticed, that when we manually binmnu-upload packages, which are already in the *same version* on debian-ports, then debian-ports ACCEPT those packages, but if we then try to apt-get-update those later on, this leads to a "size mismatch" error. I do have the feeling, that this is a problem on debian-ports. I noticed for example that reprepro usually doesn't accept packages of the same version which doesn't seem to be the case on debian-ports. So, I'm anxious, that if I start the buildd, it will happily build and upload packages which we already uploaded to debian-ports. If this happens we will get more size-mismatch errors. A trivial example: On machine buildd.debian-ports.org I run: deller@leda:~$ wb info hello . hppa * hello/hppa | hello: | Package : hello | Version : 2.8-4 | State : Needs-Build | Section : devel | Priority: source | Previous-State : | State-Change: 2013-02-18 00:03:36.782007 | CalculatedPri : 52 | component : main | Distribution: sid | Notes : out-of-date | State-Days : 300 | State-Time : 25958430 So, the package "hello" would need a rebuild according to the wanna-build database, and that would wb probably tell my buildd who then would start building/uploading it. But on http://ftp.debian-ports.org/debian/pool-hppa/main/h/hello/ you can see, that the hello-package is already uploaded at version 2.8-4 So, if my buildd now uploads the newly created hello package, I'm sure we will run again into the size-mismatch problem. Now, Aurelien mentioned last week to me, that this size-mismatch error might be because of the "apt-ftparchive cache might have been corrupted for hppa". I'm not 100% sure about that. My question here on the list would be, if you (other arch-porters) do have an idea on how I should proceed. Best solution would probably be, if the wanna-build database rescans what's in the archive already. Is this possible? Or, should I just start the buildd and see what's happening? If we then get the size-mismatch errors there is lot of manual work to fix it (unless resetting the apt-ftparchive on debian-ports would solve this). Any other ideas? Helge -- To UNSUBSCRIBE, email to debian-alpha-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/52ad8a73.6040...@gmx.de
