Re: restricted sourceless ARM uploads

[K. Richard Pixley]

I know I'm late to the party, but one big win about qemu build servers 
is that they can be instantly cloned, replicated, and shared.  We can't 
do that with real hardware.


--rich


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Wookey]

On 2006-12-20 19:58 +0100, Bill Allombert wrote:
> On Wed, Dec 20, 2006 at 12:32:25PM -0600, Bill Gatliff wrote:
> > 
> > Could you try those packages on hedges?  (You can get developer access 
> > from Wookey if you need it).  Hedges has 512MB real and 1.5GB swap.  And 
> > unlike leisner, the netwinders, or nslu2s, it's expandable if needed.
> 
> No use, this will simply thrash the box to no end. Try to build openvrml
> for example. It failed on europa after 1000 minutes of inactivity:
> 
> 

This built OK on hedges in about 6 hours. Is a binNMU needed?

(meant to get back about this earlier but forgot I had it running).

Wookey
-- 
Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK  Tel +44 (0) 1223 811679
work: http://www.aleph1.co.uk/ play: http://wookware.org/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Sam Hocevar]

On Thu, Dec 21, 2006, Wouter Verhelst wrote:

> >Pardon me sir, but can that claim that binaries built on so-called
> > "real hardware" will unquestionably run (as opposed to, if I understand
> > correcly, binaries built on an emulated platform) be backed up by any
> > facts, examples, experimentation results or scientific publication?
> 
> Binaries built on real hardware are built on a machine that uses the
> binaries built on same real hardware.

   This is not true. We have different ARM machines that use different
CPUs and hardware. Given the versatility of the ARM platform, if the
argument "building the binaries on the same machine that uses the
binaries" was valid whatsoever, it would in fact be an argument in
favour of only using a standardised emulated platform.

> I.e., if it works, at least you're sure it doesn't work because the
> compiler and the emulator are bug-compatible.

   But you're sure that it's not because the compiler and the CPU are
bug-compatible?

Regards,
-- 
Sam.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Wouter Verhelst]

On Wed, Dec 20, 2006 at 06:50:12PM +0100, Sam Hocevar wrote:
> On Wed, Dec 20, 2006, Bill Gatliff wrote:
> 
> > For the faster arches, i.e. the ARM9 machines and above, I'm thinking 
> > that we should stick with real hardware so there's no question that the 
> > binaries will run properly.
> 
>Pardon me sir, but can that claim that binaries built on so-called
> "real hardware" will unquestionably run (as opposed to, if I understand
> correcly, binaries built on an emulated platform) be backed up by any
> facts, examples, experimentation results or scientific publication?

Binaries built on real hardware are built on a machine that uses the
binaries built on same real hardware. I.e., if it works, at least you're
sure it doesn't work because the compiler and the emulator are
bug-compatible.

-- 
 Home is where you have to wash the dishes.
  -- #debian-devel, Freenode, 2004-09-22


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Kevin Mark]

On Wed, Dec 20, 2006 at 06:40:06PM +0100, Aurelien Jarno wrote:
> Wookey a écrit :
> > On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
> >> Hi,
> >>
> >> For those who don't know, I have setup 8 emulated ARM build daemons and
> >> started to upload packages. To know why and for more information, see
> >> [1].
> > 
> > Very impressive piece of work aurelien! We ought to discuss if there
> > is any significant reason not to use qemu 'machines' instead of actual
> > hardware for slower arches.
> > 
> 
> BTW, I don't think ARM needs emulated build machines. It's just that
> it's the fastest ARM machine I found. My two NSLU2 won't have been enough.
> 
> Also it seems the current ARM build machines altogether are fast enough
> to build all the packages (except for building big packages like the
> kernel). But a faster machine would mean that the Vancouver requirements
> are satisfied, and also less machines to handle, so less work for the
> ARM build daemon maintainer.
Hi *,
that brings up an interesting issue wrt the vancouver memorandum: it
didn't address software emulation or virtualization. Can qemu and xen be
shown to be reliable for buildd purposes? Does the aforementioned
document need to be modified for these?
cheers,
Kev
-- 
|  .''`.  == Debian GNU/Linux == |   my web site:   |
| : :' :  The  Universal | debian.home.pipeline.com |
| `. `'  Operating System| go to counter.li.org and |
|   `-http://www.debian.org/ |be counted! #238656   |
| my keysever: pgp.mit.edu   | my NPO: cfsg.org |


signature.asc
Description: Digital signature

Re: restricted sourceless ARM uploads

[Russ Allbery]

Aurelien Jarno <[EMAIL PROTECTED]> writes:

> * Also those package (and they are plenty others), could be tagged
>   not-for-us. This would spare some build daemon time, and would ease to
>   see which packages have really failed to build or not.
[...]
>openafs_1.4.2-4

As the OpenAFS co-maintainer, I can confirm that would be the correct
action to take.  OpenAFS tries to fail its build fairly quickly with a
clear error message, but it really should just be tagged not-for-us on arm
(and on mips and mipsel).  I tried contacting the Packages-arch-specific
maintainers a few times about this a while back without a response.

-- 
Russ Allbery ([EMAIL PROTECTED])   


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Aurelien Jarno]

Mirco Bauer a écrit :
> Hi Aurelien,
> 
> I can give some details about the Mono related packages that failed to
> build before but worked for you. I reply inline.
> 
> On Wed, 2006-12-20 at 17:39 +0100, Aurelien Jarno wrote:
>> * Packages that now build fine:
>>blam_1.8.3-2 (for 40 days)

Oops, there is a mistake (copy & paste abuse), this should have been 9
days, as other mono related packages.

> blam failed because of a runtime bug (which is specific to the arm port)
> in Mono, which can be indentified by throwing random
> System.IO.FileNotFoundException, which was fixed in Mono 1.2.2.1 (see
> #394418), so a simple rebuild solves this kind of FTBFS on ARM for Mono
> based software. Though there are remaining problems still with Mono and
> netwinder, which is why the bug isn't closed yet.

When you say "netwinder", could you reproduce it on others netwinder
machines than the build daemon called that way?

FYI this build daemon seems to have some problems (missing files).

-- 
  .''`.  Aurelien Jarno | GPG: 1024D/F1BCDB73
 : :' :  Debian developer   | Electrical Engineer
 `. `'   [EMAIL PROTECTED] | [EMAIL PROTECTED]
   `-people.debian.org/~aurel32 | www.aurel32.net


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Mirco Bauer]

Hi Aurelien,

I can give some details about the Mono related packages that failed to
build before but worked for you. I reply inline.

On Wed, 2006-12-20 at 17:39 +0100, Aurelien Jarno wrote:
> * Packages that now build fine:
>blam_1.8.3-2 (for 40 days)
blam failed because of a runtime bug (which is specific to the arm port)
in Mono, which can be indentified by throwing random
System.IO.FileNotFoundException, which was fixed in Mono 1.2.2.1 (see
#394418), so a simple rebuild solves this kind of FTBFS on ARM for Mono
based software. Though there are remaining problems still with Mono and
netwinder, which is why the bug isn't closed yet.

>evolution-sharp_0.11.1-3 (for 9 days)
Same thing here as for blam

>gcc-h8300-hms_4.1.1-3 (for 7 days)
>hdbc-odbc_1.0.1.1 (for 40 days) 
>hdbc-missingh_1.0.1.1 (for 40 days)
>njb-sharp_0.3.0-1 (for 9 days)
Same thing as for blam

>muine_0.8.5-1.1 (for 9 days)
This one seems to fail trying to generate some WDSL stuff, that tries to
use the internet, which is not always available on some buildds, but was
on your machine seems like.

Can't say anything about the other packages.

-- 
Regards,

Mirco 'meebey' Bauer

PGP-Key:
http://keyserver.noreply.org/pks/lookup?op=get&search=0xEEF946C8

-BEGIN GEEK CODE BLOCK-
Version: 3.12
GIT d s-:+ a-- C++ UL$ P L++$>+++$ E- W+++$ N o? K- w++>! O M-
V? PS
PE+ Y- PGP++ t 5+ X++ R tv+ b+ DI? D+ G>++ e h! r->++ y?
--END GEEK CODE BLOCK--


signature.asc
Description: This is a digitally signed message part

Re: restricted sourceless ARM uploads

[Bill Allombert]

On Wed, Dec 20, 2006 at 12:32:25PM -0600, Bill Gatliff wrote:
> 
> Could you try those packages on hedges?  (You can get developer access 
> from Wookey if you need it).  Hedges has 512MB real and 1.5GB swap.  And 
> unlike leisner, the netwinders, or nslu2s, it's expandable if needed.

No use, this will simply thrash the box to no end. Try to build openvrml
for example. It failed on europa after 1000 minutes of inactivity:



> But on that point, I've never had any issues with either 
> distcc+crossgcc--- which I've tested extensively--- or QEMU.  But 

Neither I have (I use distcc+crossgcc on top of aranym quite
extensively) so I don't see any reason not to use that if that help
the port to be releasable.

> forcing the use of real hardware wherever possible means (a) you know 
> for sure, and (b) you have to keep your real hardware maintained.  Those 
> seem like good things.

They are sure good thing but not at the price of the port not
being released which is the issue here.

Cheers,
-- 
Bill. <[EMAIL PROTECTED]>

Imagine a large blue swirl here. 


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Bill Gatliff]

Bill:



A very basic reason is that some packages require 1GB of RAM to build
in finite time and there are no arm and m68k buildd with that amount of
RAM.
 



Could you try those packages on hedges?  (You can get developer access 
from Wookey if you need it).  Hedges has 512MB real and 1.5GB swap.  And 
unlike leisner, the netwinders, or nslu2s, it's expandable if needed.



An alternative is to use distcc+crosscc to a distccd server with 1GB of RAM.
 



I've done this before (rebuilt X that way, just as a test!), but it has 
similar concerns to the QEMU approach.


But on that point, I've never had any issues with either 
distcc+crossgcc--- which I've tested extensively--- or QEMU.  But 
forcing the use of real hardware wherever possible means (a) you know 
for sure, and (b) you have to keep your real hardware maintained.  Those 
seem like good things.




b.g.

--
Bill Gatliff
[EMAIL PROTECTED]


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Aurelien Jarno]

Aurelien Jarno a écrit :
> * Packages that failed to build because the build daemon netwinder is
>   fucked for weeks wrt to /usr/lib/libtasn1.so.3.0.6
>cpufire-applet_1.2-2 
>gnome-session_2.14.3-4 

The situation is actually worse than what I expected. This build daemon
seems to loose files from its chroot. I don't have access to it, but
from the build logs I can say that /sbin/MAKEDEV [1] and
/usr/lib/libtasn1.so.3.0.6 [2] are missing.

Other files may be missing. Given that a lot of packages are using
autoconf to detect available libraries, some packages built by this
build daemon may have some disable functionalities.

Bye,
Aurelien

[1]
http://buildd.debian.org/build.php?pkg=kdebase&arch=arm&ver=4:3.5.5a.dfsg.1-4

[2]
http://buildd.debian.org/fetch.cgi?&pkg=gnome-session&ver=2.14.3-4&arch=arm&stamp=1166563605&file=log

-- 
  .''`.  Aurelien Jarno | GPG: 1024D/F1BCDB73
 : :' :  Debian developer   | Electrical Engineer
 `. `'   [EMAIL PROTECTED] | [EMAIL PROTECTED]
   `-people.debian.org/~aurel32 | www.aurel32.net


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Sam Hocevar]

On Wed, Dec 20, 2006, Bill Gatliff wrote:

> For the faster arches, i.e. the ARM9 machines and above, I'm thinking 
> that we should stick with real hardware so there's no question that the 
> binaries will run properly.

   Pardon me sir, but can that claim that binaries built on so-called
"real hardware" will unquestionably run (as opposed to, if I understand
correcly, binaries built on an emulated platform) be backed up by any
facts, examples, experimentation results or scientific publication?

Kindest regards,
-- 
Sam.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Bill Allombert]

On Wed, Dec 20, 2006 at 05:05:21PM +, Wookey wrote:
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
> > Hi,
> > 
> > For those who don't know, I have setup 8 emulated ARM build daemons and
> > started to upload packages. To know why and for more information, see
> > [1].
> 
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.

A very basic reason is that some packages require 1GB of RAM to build
in finite time and there are no arm and m68k buildd with that amount of
RAM.

An alternative is to use distcc+crosscc to a distccd server with 1GB of
RAM.

Cheers,
-- 
Bill. <[EMAIL PROTECTED]>

Imagine a large red swirl here.


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Aurelien Jarno]

Wookey a écrit :
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
>> Hi,
>>
>> For those who don't know, I have setup 8 emulated ARM build daemons and
>> started to upload packages. To know why and for more information, see
>> [1].
> 
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.
> 

BTW, I don't think ARM needs emulated build machines. It's just that
it's the fastest ARM machine I found. My two NSLU2 won't have been enough.

Also it seems the current ARM build machines altogether are fast enough
to build all the packages (except for building big packages like the
kernel). But a faster machine would mean that the Vancouver requirements
are satisfied, and also less machines to handle, so less work for the
ARM build daemon maintainer.

What I wanted to demonstrate, is that the percentage of package built by
an architecture does depends on the speed of the build machines, but
also on how they are administrated.

-- 
  .''`.  Aurelien Jarno | GPG: 1024D/F1BCDB73
 : :' :  Debian developer   | Electrical Engineer
 `. `'   [EMAIL PROTECTED] | [EMAIL PROTECTED]
   `-people.debian.org/~aurel32 | www.aurel32.net


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Bill Gatliff]

Wookey wrote:


On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
 


Hi,

For those who don't know, I have setup 8 emulated ARM build daemons and
started to upload packages. To know why and for more information, see
[1].
   



Very impressive piece of work aurelien! We ought to discuss if there
is any significant reason not to use qemu 'machines' instead of actual
hardware for slower arches.
 



As long as there's the occasional test on actual hardware, I don't have 
a problem with it.


For the faster arches, i.e. the ARM9 machines and above, I'm thinking 
that we should stick with real hardware so there's no question that the 
binaries will run properly.



How fast is each of your build machines in comparison to existing
buildds?

The offered Hedges machine is more than twice as fast as existing
buildds and really ought to be brought into the network - it has been
offered for about 2 months now. I will mail DSA again to see if anyone
will tell what needs to be done next, by whom, and when. It is very
rude to people making such offers to give no response at all for such
a long time. Fortunately bill seems very tolerant of Debian's foibles,
 



What choice do I have? I'm not a DD, so it isn't like I have any 
authority to do otherwise. :)




and we can at least use it as a private developer-access machine in
the meantime.
 



Indeed.



b.g.

--
Bill Gatliff
[EMAIL PROTECTED]


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Aurelien Jarno]

Wookey a écrit :
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
>> Hi,
>>
>> For those who don't know, I have setup 8 emulated ARM build daemons and
>> started to upload packages. To know why and for more information, see
>> [1].
> 
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.

I think we should include Paul Brook in the discussion, as he written a
lot of parts of the ARM emulation, and is also active in the Debian ARM
port.

Anyway I will say that if we can have fast machine (and there are fast
ARM machines), it is better than a few not to slow emulated ARM machine.
But that machine should be accepted by the DSA. And here is the problem.

> How fast is each of your build machines in comparison to existing
> buildds?

For what I saw they are a bit faster than the Debian build machines. You
can expect to have a ARM v5 running at around 350 MHz. And with plenty
of RAM (256 MB in my case), so that make a huge difference on some packages.

> The offered Hedges machine is more than twice as fast as existing
> buildds and really ought to be brought into the network - it has been
> offered for about 2 months now. I will mail DSA again to see if anyone
> will tell what needs to be done next, by whom, and when. It is very
> rude to people making such offers to give no response at all for such
> a long time. Fortunately bill seems very tolerant of Debian's foibles,
> and we can at least use it as a private developer-access machine in
> the meantime.
> 
>> Note that I would have been happy to not start those build daemons, but
>> given the way the arm build daemons are administrated, something had to 
>> be done.
> 
> Perhaps you are right. Let us all try to keep this discussion
> constructive. 
>  
>> Anyway, I have been able to fix a few packages that were not built
>> successfully. For the others, I have extracted from wanna-build a list
>> of package that build successfully, but have failed to build for 
>> various reasons. 
> 
> 
> 
> Can you update the armbuildd wiki page with this? (I expect you are going to
> anyway).

Yes, I will do that later today.

Bye,
Aurelien

-- 
  .''`.  Aurelien Jarno | GPG: 1024D/F1BCDB73
 : :' :  Debian developer   | Electrical Engineer
 `. `'   [EMAIL PROTECTED] | [EMAIL PROTECTED]
   `-people.debian.org/~aurel32 | www.aurel32.net


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: restricted sourceless ARM uploads

[Wookey]

On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
> Hi,
> 
> For those who don't know, I have setup 8 emulated ARM build daemons and
> started to upload packages. To know why and for more information, see
> [1].

Very impressive piece of work aurelien! We ought to discuss if there
is any significant reason not to use qemu 'machines' instead of actual
hardware for slower arches.

How fast is each of your build machines in comparison to existing
buildds?

The offered Hedges machine is more than twice as fast as existing
buildds and really ought to be brought into the network - it has been
offered for about 2 months now. I will mail DSA again to see if anyone
will tell what needs to be done next, by whom, and when. It is very
rude to people making such offers to give no response at all for such
a long time. Fortunately bill seems very tolerant of Debian's foibles,
and we can at least use it as a private developer-access machine in
the meantime.

> Note that I would have been happy to not start those build daemons, but
> given the way the arm build daemons are administrated, something had to 
> be done.

Perhaps you are right. Let us all try to keep this discussion
constructive. 
 
> Anyway, I have been able to fix a few packages that were not built
> successfully. For the others, I have extracted from wanna-build a list
> of package that build successfully, but have failed to build for 
> various reasons. 



Can you update the armbuildd wiki page with this? (I expect you are going to
anyway).

Thanx you for your efforts on the arm port over the last few months -
they have been extremely helpful.

Can we make aurel32 an arm buildd maintainer? I think he has shown
both competence and dedication this year. I'm sure having more than
one maintainer would be helpful. Aurel - do you want to commit to this
task?

Wookey
-- 
Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK  Tel +44 (0) 1223 811679
work: http://www.aleph1.co.uk/ play: http://wookware.org/


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]