Re: Future of debian-cd

[Richard Atterer]

On Wed, Jan 07, 2004 at 11:06:55PM +, Steve McIntyre wrote:
> Something that Phil and I discussed a while ago - it should be
> feasible to produce jigdo images of CDs directly from Packages,

jigdo needs each file's md5sum and another, special checksum of the first
1k of data of each file. Those checksums would have to be made available
for download on a server for this to work.

But doesn't some architecture do something really dirty to make CDs
bootable - overwriting part of the image output by mkisofs, or similar?

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯

Re: Future of debian-cd

[Richard Atterer]

On Wed, Jan 07, 2004 at 11:06:55PM +, Steve McIntyre wrote:
> Something that Phil and I discussed a while ago - it should be
> feasible to produce jigdo images of CDs directly from Packages,

jigdo needs each file's md5sum and another, special checksum of the first
1k of data of each file. Those checksums would have to be made available
for download on a server for this to work.

But doesn't some architecture do something really dirty to make CDs
bootable - overwriting part of the image output by mkisofs, or similar?

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Future of debian-cd

[Delian Krustev]

On Wednesday 07 January 2004 15:16, Raphael Hertzog wrote:
> Hello everyone,
>
> many of you already noticed that debian-cd is a bit complicated.
> And it tends to get worse since we keep doing small modifications
> everywhere without a global logic.
>
> Thus I think that debian-cd needs to be partially rewritten.
>
> Here are the main critics that I have :
> - complicated shell code in Makefile is ugly, it was supposed to
>   not need to be modified, and in fact many parts of the Makefile
>   were modified for various purposes

Richard idea to split it in dirs sounds good. The approaches of multiple
Makefiles/scrips which invoke each other seem similar in human efforts
to me. I would probably prefer multiple Makefiles thou.

> - you can't reasonnably build potato cd with current debian-cd because
>   of those "infrastructural change" to a CD (inclusion of udeb for
>   example) which were not expected at the time of my initial rewrite
>   of debian-cd. I still managed to keep most of the compatibility
>   but at the cost of more code complexity.
> - we have too many imbricated options which do not make sense to many
>   of us, it's time to see if we shouldn't get rid of some of them
>   (or at least hide them somewhere deeper than CONF.sh)
> - the official support of businesscard cd/netinst cd/dvd is not good, most
>   people are not able to build such CD with debian-cd because they don't
>   know how the magic combination of parameters.
> - it's too much of black magick to know the good size parameters to give
>
> In a new design, I would like to have :
> - a MEDIA_TYPE input variable that would tell me :
>   1 - businesscard CD
>   2 - netinst CD
>   3 - usual CD
>   4 - DVD
>   The size would be taken from a default value (if not overriden).
> - a system of profile that would describe how to build a particular kind
>   of ISO
> - a system of profile for debian-installer, so that we can easily
>   customize the udebs used by the debian installer
>
> The new design should still reuse most of the existing code, not
> everything needs to be thrown. In fact I think that most should be kept
> but cleaned from the cruft and reorganized in a different way.
>
> Now, I'd like to hear from you :
> - what are your (constructive) critics against debian-cd ?

It was quite difficult making the debian-cd cvs version running on redhat 7.3.
I don't know how the guy that wrote he's using debian-cd to build
unstable managed to do that. Certainly not without modifications.
I've also managed to build sid images but not a bootable ones, since
there were no boot images or tasks  for sid in the tree(honestly, 
I've forgotten what the exact cause of that problem was).

I think debian-cd should run on any unix/linux. Some binary packages
could be provided for OSes other than debian. Not everyone keeping
a local debian mirror is running debian on that box.

I don't find jidgo suitable for "main way of distributing Debian CD images".
I've posted about that and Richard didn't succeed to argue about the
jidgo advantages over the rsync/debian-cd approach. The only use of jidgo
I find is to fetch/update the isos when the preferred mirror is not
providing an rsync access, and there's no other useful mirror nearby.

A word about the debian installer. I've tried to install sid on a recent
box with an old monitor. It was a 12" black and white with very poor
timings. In fact it is capable of running text mode and e.g. [EMAIL PROTECTED]
Unfortunately debian installer changes the text mode timings and the
image was screwed up.

As for the scripting language I would choose perl (not willing to argue).


> - what design ideas do you have for a possible rewrite ?
> - what do you think of what i've written above ?
> - how do you think we should proceed ? should I start from a completely
>   new repository and import only what I want to keep ? should we work in
>   a branch of the actual repository ?

The main development line usually goes in the HEAD. No matter how drastic the
changes are. If You don't do so You will have to merge the changes back 
in the HEAD at some point in the future, which I find is good to be avoided.

>
> BTW, as you may have noticed I have less & less time for debian-cd, and
> I'd like to have some help in the long term to help me maintain it. With
> a (partial) rewrite, it would be the perfect time for someone to step up
> and get familiar with the code ...
>
> Cheers,


Cheers,
Delian Krustev

Re: Future of debian-cd

[Delian Krustev]

On Wednesday 07 January 2004 15:16, Raphael Hertzog wrote:
> Hello everyone,
>
> many of you already noticed that debian-cd is a bit complicated.
> And it tends to get worse since we keep doing small modifications
> everywhere without a global logic.
>
> Thus I think that debian-cd needs to be partially rewritten.
>
> Here are the main critics that I have :
> - complicated shell code in Makefile is ugly, it was supposed to
>   not need to be modified, and in fact many parts of the Makefile
>   were modified for various purposes

Richard idea to split it in dirs sounds good. The approaches of multiple
Makefiles/scrips which invoke each other seem similar in human efforts
to me. I would probably prefer multiple Makefiles thou.

> - you can't reasonnably build potato cd with current debian-cd because
>   of those "infrastructural change" to a CD (inclusion of udeb for
>   example) which were not expected at the time of my initial rewrite
>   of debian-cd. I still managed to keep most of the compatibility
>   but at the cost of more code complexity.
> - we have too many imbricated options which do not make sense to many
>   of us, it's time to see if we shouldn't get rid of some of them
>   (or at least hide them somewhere deeper than CONF.sh)
> - the official support of businesscard cd/netinst cd/dvd is not good, most
>   people are not able to build such CD with debian-cd because they don't
>   know how the magic combination of parameters.
> - it's too much of black magick to know the good size parameters to give
>
> In a new design, I would like to have :
> - a MEDIA_TYPE input variable that would tell me :
>   1 - businesscard CD
>   2 - netinst CD
>   3 - usual CD
>   4 - DVD
>   The size would be taken from a default value (if not overriden).
> - a system of profile that would describe how to build a particular kind
>   of ISO
> - a system of profile for debian-installer, so that we can easily
>   customize the udebs used by the debian installer
>
> The new design should still reuse most of the existing code, not
> everything needs to be thrown. In fact I think that most should be kept
> but cleaned from the cruft and reorganized in a different way.
>
> Now, I'd like to hear from you :
> - what are your (constructive) critics against debian-cd ?

It was quite difficult making the debian-cd cvs version running on redhat 7.3.
I don't know how the guy that wrote he's using debian-cd to build
unstable managed to do that. Certainly not without modifications.
I've also managed to build sid images but not a bootable ones, since
there were no boot images or tasks  for sid in the tree(honestly, 
I've forgotten what the exact cause of that problem was).

I think debian-cd should run on any unix/linux. Some binary packages
could be provided for OSes other than debian. Not everyone keeping
a local debian mirror is running debian on that box.

I don't find jidgo suitable for "main way of distributing Debian CD images".
I've posted about that and Richard didn't succeed to argue about the
jidgo advantages over the rsync/debian-cd approach. The only use of jidgo
I find is to fetch/update the isos when the preferred mirror is not
providing an rsync access, and there's no other useful mirror nearby.

A word about the debian installer. I've tried to install sid on a recent
box with an old monitor. It was a 12" black and white with very poor
timings. In fact it is capable of running text mode and e.g. [EMAIL PROTECTED]
Unfortunately debian installer changes the text mode timings and the
image was screwed up.

As for the scripting language I would choose perl (not willing to argue).


> - what design ideas do you have for a possible rewrite ?
> - what do you think of what i've written above ?
> - how do you think we should proceed ? should I start from a completely
>   new repository and import only what I want to keep ? should we work in
>   a branch of the actual repository ?

The main development line usually goes in the HEAD. No matter how drastic the
changes are. If You don't do so You will have to merge the changes back 
in the HEAD at some point in the future, which I find is good to be avoided.

>
> BTW, as you may have noticed I have less & less time for debian-cd, and
> I'd like to have some help in the long term to help me maintain it. With
> a (partial) rewrite, it would be the perfect time for someone to step up
> and get familiar with the code ...
>
> Cheers,


Cheers,
Delian Krustev


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Future of debian-cd

[Richard Atterer]

On Wed, Jan 07, 2004 at 02:24:14PM +0100, [EMAIL PROTECTED] wrote:
> Since debian-cd is no longer available to the public, it is very
> difficult to test currently. Maybe someone could provide a current .deb
> package for testing or unstable, so people can actually try out the most
> recent version.

I've uploaded a tarball with a CVS snapshot to 


> Are the "maintainers" of the "testing" and "unstable" CDs/DVD images
> still using debian-cd or something else ?

debian-cd AFAIK.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯

Re: Future of debian-cd

[Richard Atterer]

On Wed, Jan 07, 2004 at 02:24:14PM +0100, [EMAIL PROTECTED] wrote:
> Since debian-cd is no longer available to the public, it is very
> difficult to test currently. Maybe someone could provide a current .deb
> package for testing or unstable, so people can actually try out the most
> recent version.

I've uploaded a tarball with a CVS snapshot to 


> Are the "maintainers" of the "testing" and "unstable" CDs/DVD images
> still using debian-cd or something else ?

debian-cd AFAIK.

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

TFH> I don't think the python code here looks better at all. It's
TFH> possible to write both readable and unreadable python or perl.

you're right :-) In any language you can write ugly code, but I found
it a bit easier to wirte "readable" (what ever that means *g*) code in
python, mostly because of the structuring via indents. But this is
only my point of view :-)

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//RcfvvmCkIIgH8QRAhWkAJ9WJT5kxhyL8i3v6BDaD3+xZCEiQwCdGV9q
2zowCEvvz5itNnIhvNPxKXI=
=2e37
-END PGP SIGNATURE-

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello Steve,

Thursday, January 8, 2004, 12:06:55 AM, hast Du geschrieben:

SM> Something that Phil and I discussed a while ago - it should be
SM> feasible to produce jigdo images of CDs directly from Packages,
SM> Release and/or Sources files without needing any of the rest of the
SM> mirror. With the patch I've made to mkisofs and some extra work in
SM> jigdo itself, we should be able to drop production times for images
SM> from multiple hours down to a few minutes...

SM> Sound interesting? :-)

Yes it does ;-) For many people I'm building sarge and testing images
and every run takes hours to finish, so if here it's possible to speed
things up, that would be great.

By the way: is it possible to get recent cvs versions from cvs?

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//RRwvvmCkIIgH8QRAn/QAKCDWAfuAae6MXuDZDKcwwFBfSnoWgCghNkj
1+jNUMKmuvijnxHQmf0QnmQ=
=pDKs
-END PGP SIGNATURE-

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

TFH> I don't think the python code here looks better at all. It's
TFH> possible to write both readable and unreadable python or perl.

you're right :-) In any language you can write ugly code, but I found
it a bit easier to wirte "readable" (what ever that means *g*) code in
python, mostly because of the structuring via indents. But this is
only my point of view :-)

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//RcfvvmCkIIgH8QRAhWkAJ9WJT5kxhyL8i3v6BDaD3+xZCEiQwCdGV9q
2zowCEvvz5itNnIhvNPxKXI=
=2e37
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello Steve,

Thursday, January 8, 2004, 12:06:55 AM, hast Du geschrieben:

SM> Something that Phil and I discussed a while ago - it should be
SM> feasible to produce jigdo images of CDs directly from Packages,
SM> Release and/or Sources files without needing any of the rest of the
SM> mirror. With the patch I've made to mkisofs and some extra work in
SM> jigdo itself, we should be able to drop production times for images
SM> from multiple hours down to a few minutes...

SM> Sound interesting? :-)

Yes it does ;-) For many people I'm building sarge and testing images
and every run takes hours to finish, so if here it's possible to speed
things up, that would be great.

By the way: is it possible to get recent cvs versions from cvs?

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//RRwvvmCkIIgH8QRAn/QAKCDWAfuAae6MXuDZDKcwwFBfSnoWgCghNkj
1+jNUMKmuvijnxHQmf0QnmQ=
=pDKs
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Future of debian-cd

[Steve McIntyre]

On Wed, Jan 07, 2004 at 11:20:58AM -0500, Joey Hess wrote:
>Raphael Hertzog wrote:
>> Here are the main critics that I have :
>
>Let me add my main problem:
>
>- Requires a full debian mirror to work.
>
>IMHO, it would be much better to follow the lead of how d-i builds its
>boot images, by using apt to download (u)debs. debian-cd would also need
>to use wget or something for the non-deb components, but it should be
>possible to use it with nothing but a network connection, and caching.

Something that Phil and I discussed a while ago - it should be
feasible to produce jigdo images of CDs directly from Packages,
Release and/or Sources files without needing any of the rest of the
mirror. With the patch I've made to mkisofs and some extra work in
jigdo itself, we should be able to drop production times for images
from multiple hours down to a few minutes...

Sound interesting? :-)

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Support the Campaign for Audiovisual Free Expression: http://www.eff.org/cafe/


signature.asc
Description: Digital signature

Re: Future of debian-cd

[Steve McIntyre]

On Wed, Jan 07, 2004 at 11:20:58AM -0500, Joey Hess wrote:
>Raphael Hertzog wrote:
>> Here are the main critics that I have :
>
>Let me add my main problem:
>
>- Requires a full debian mirror to work.
>
>IMHO, it would be much better to follow the lead of how d-i builds its
>boot images, by using apt to download (u)debs. debian-cd would also need
>to use wget or something for the non-deb components, but it should be
>possible to use it with nothing but a network connection, and caching.

Something that Phil and I discussed a while ago - it should be
feasible to produce jigdo images of CDs directly from Packages,
Release and/or Sources files without needing any of the rest of the
mirror. With the patch I've made to mkisofs and some extra work in
jigdo itself, we should be able to drop production times for images
from multiple hours down to a few minutes...

Sound interesting? :-)

-- 
Steve McIntyre, Cambridge, UK.[EMAIL PROTECTED]
Support the Campaign for Audiovisual Free Expression: http://www.eff.org/cafe/


signature.asc
Description: Digital signature

Re: Future of debian-cd

[Tollef Fog Heen]

* Jan Kesten 

| Here is the perl way of opening a file, reading it and sorting the
| lines into a new file:

#! /usr/bin/perl -w

use strict;

my $file = shift;
open IN, "<$file" or die "Can't open $file: $!";
open OUT, ">$file.sorted" or die "Can't open $file.sorted: $!";

print OUT sort ;

This is fairly readable to me at least; yes, it's possible to write it
shorter, and it's possible to write it without strict, but it's a nice
safety net.

| And the python way looks like this:

[...]


| Some other thing is that python knows about classes :-) But I do not
| want to start a discussion about pro perl or python :-)

I don't think the python code here looks better at all.  It's possible
to write both readable and unreadable python or perl.

-- 
Tollef Fog Heen,''`.
UNIX is user friendly, it's just picky about who its friends are  : :' :
  `. `' 
`-  

Re: Future of debian-cd

[Hans Ginzel]

On Wed, Jan 07, 2004 at 06:00:51PM +0100, Jan Kesten wrote:
> # Datei lesen
> open DATEI, "<$dateiname" or die "$dateiname laesst sich nicht oeffnen\n";

   Use the $! variable (man perlvar; perldoc perlvar; www.perldoc.com):
open DATEI, "<$dateiname" or die "Datei `$dateiname' laesst sich nicht oeffnen: 
$!\n";

> @zeilen =   or die "$dateiname ist nicht lesbar\n"
> close DATEI;

   close DATEI or warn "Cannot close file `$dateiname': $!\n";

> # sortieren

   You can sort just by reading:

@sorted_zeilen = sort  or die $!;

> -  SNIP 
>
> And the python way looks like this:
>
> # Datei lesen
> try:
> datei = open(dateiname, 'r')
> zeilen = datei.readlines()
> datei.close()
> except IOError:
> print dateiname, "ist nicht lesbar"
> sys.exit( 1 )

   That's not what is above in perl. I understand especially: you
are saying that Python has exceptions. But Perl also has! Read about
`eval' in man perlfunc or `perldoc -f eval'. The advantidge of using
"die" for each part of code is that you can give more precise error
message, even more if combined with the `$!' variable. Above Python code
would be almost ekvivalently rewritten into Perl:

eval {
open DATEI, "<$dateiname" or die $!;
@zeilen =  or die $!;
close DATEI;
}
die "Datei `$dateiname' ist nicht lesbar: [EMAIL PROTECTED]" if $@;

> Some other thing is that python knows about classes :-) But I do not

   Perl also knows about classes (perldoc -f bless; man perlboot,
perltoot, perlobj).

> want to start a discussion about pro perl or python :-)

   Nor want I, but be precise.

Best regards

Hans Ginzel

Re: Future of debian-cd

[Raphael Hertzog]

[ I cced [EMAIL PROTECTED] in my initial mail in order to have a paragraph
about debian-cd in DWN next week - no need to continue CCing them even
if my Mail-Followup-To included them ]

Le Wed, Jan 07, 2004 at 03:59:52PM +0100, Richard Atterer écrivait:
> I think the idea of using a Makefile at all wasn't that good in retrospect. 

In some aspects it was interesting, I liked to be able to avoid redoing
stuff which were already done when I corrected something by hand. But
that's it.

> Maybe it would be nicer to use "run-parts": A top-level script executes
> everything in a "stages" directory in alphabetical order. The individual
> stages could then have further directories of their own where
> substage-scripts for that stage of the process are stored. This would get 
> rid of all that "hook" ugliness...
> 
> There could be different top-level "stages" directories for the different 
> CD flavours, with symlinks to scripts which are identical between any two 
> of them. This has the advantage that you can fork code any time; instead of 
> linking from the sarge version to the woody version, just make a copy of 
> the script, then make changes/hacks which are woody-only.
> 
> Each top-level "stages" dir could also come with configuration files 
> preconfigured for that particular CD flavour.
> 
> Well, something like that... I don't know if the above would work. :)

It's exactly something like that that I had in mind when I spoke of
"profile" in my initial mail. And I have the same problem than you, it
looks like nice but I didn't took the time to see it it fits as well as
I hope.

Cheers,
-- 
Raphaël Hertzog -+- http://www.ouaza.com
Formation Linux et logiciel libre : http://www.logidee.com
Earn money with free software: http://www.geniustrader.org

Re: Future of debian-cd

[Tollef Fog Heen]

* Jan Kesten 

| Here is the perl way of opening a file, reading it and sorting the
| lines into a new file:

#! /usr/bin/perl -w

use strict;

my $file = shift;
open IN, "<$file" or die "Can't open $file: $!";
open OUT, ">$file.sorted" or die "Can't open $file.sorted: $!";

print OUT sort ;

This is fairly readable to me at least; yes, it's possible to write it
shorter, and it's possible to write it without strict, but it's a nice
safety net.

| And the python way looks like this:

[...]


| Some other thing is that python knows about classes :-) But I do not
| want to start a discussion about pro perl or python :-)

I don't think the python code here looks better at all.  It's possible
to write both readable and unreadable python or perl.

-- 
Tollef Fog Heen,''`.
UNIX is user friendly, it's just picky about who its friends are  : :' :
  `. `' 
`-  


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Future of debian-cd

[Hans Ginzel]

On Wed, Jan 07, 2004 at 06:00:51PM +0100, Jan Kesten wrote:
> # Datei lesen
> open DATEI, "<$dateiname" or die "$dateiname laesst sich nicht oeffnen\n";

   Use the $! variable (man perlvar; perldoc perlvar; www.perldoc.com):
open DATEI, "<$dateiname" or die "Datei `$dateiname' laesst sich nicht oeffnen: $!\n";

> @zeilen =   or die "$dateiname ist nicht lesbar\n"
> close DATEI;

   close DATEI or warn "Cannot close file `$dateiname': $!\n";

> # sortieren

   You can sort just by reading:

@sorted_zeilen = sort  or die $!;

> -  SNIP 
>
> And the python way looks like this:
>
> # Datei lesen
> try:
> datei = open(dateiname, 'r')
> zeilen = datei.readlines()
> datei.close()
> except IOError:
> print dateiname, "ist nicht lesbar"
> sys.exit( 1 )

   That's not what is above in perl. I understand especially: you
are saying that Python has exceptions. But Perl also has! Read about
`eval' in man perlfunc or `perldoc -f eval'. The advantidge of using
"die" for each part of code is that you can give more precise error
message, even more if combined with the `$!' variable. Above Python code
would be almost ekvivalently rewritten into Perl:

eval {
open DATEI, "<$dateiname" or die $!;
@zeilen =  or die $!;
close DATEI;
}
die "Datei `$dateiname' ist nicht lesbar: [EMAIL PROTECTED]" if $@;

> Some other thing is that python knows about classes :-) But I do not

   Perl also knows about classes (perldoc -f bless; man perlboot,
perltoot, perlobj).

> want to start a discussion about pro perl or python :-)

   Nor want I, but be precise.

Best regards

Hans Ginzel


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Future of debian-cd

[Raphael Hertzog]

[ I cced [EMAIL PROTECTED] in my initial mail in order to have a paragraph
about debian-cd in DWN next week - no need to continue CCing them even
if my Mail-Followup-To included them ]

Le Wed, Jan 07, 2004 at 03:59:52PM +0100, Richard Atterer écrivait:
> I think the idea of using a Makefile at all wasn't that good in retrospect. 

In some aspects it was interesting, I liked to be able to avoid redoing
stuff which were already done when I corrected something by hand. But
that's it.

> Maybe it would be nicer to use "run-parts": A top-level script executes
> everything in a "stages" directory in alphabetical order. The individual
> stages could then have further directories of their own where
> substage-scripts for that stage of the process are stored. This would get 
> rid of all that "hook" ugliness...
> 
> There could be different top-level "stages" directories for the different 
> CD flavours, with symlinks to scripts which are identical between any two 
> of them. This has the advantage that you can fork code any time; instead of 
> linking from the sarge version to the woody version, just make a copy of 
> the script, then make changes/hacks which are woody-only.
> 
> Each top-level "stages" dir could also come with configuration files 
> preconfigured for that particular CD flavour.
> 
> Well, something like that... I don't know if the above would work. :)

It's exactly something like that that I had in mind when I spoke of
"profile" in my initial mail. And I have the same problem than you, it
looks like nice but I didn't took the time to see it it fits as well as
I hope.

Cheers,
-- 
Raphaël Hertzog -+- http://www.ouaza.com
Formation Linux et logiciel libre : http://www.logidee.com
Earn money with free software: http://www.geniustrader.org


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

bid> Well, I always had my doubts about Python. Of course, I have done
bid> next to nothing with Python and quite some code in Perl, so quite
bid> different experiences ;)

bid> Of course Perl has other problems like readability of code and
bid> soon compatibility problems when they switch from 5 to 6.

That's why I prefered python ;-) It's very easy to read and
understand: Look here... some thing I found on a website - it is in
german but I think it doesn't matter if I do not translate it:

Here is the perl way of opening a file, reading it and sorting the
lines into a new file:

-  SNIP 
#!/usr/bin/perl -w

# Dateiname ermitteln
$dateiname = shift;

# Datei lesen
open DATEI, "<$dateiname"  or
  die "$dateiname laesst sich nicht oeffnen\n"
@zeilen =   or
  die "$dateiname ist nicht lesbar\n"
close DATEI;

# sortieren
@zeilen = sort @zeilen;
$dateiname .= '.sortiert';

# Datei schreiben
open DATEI, ">$dateiname"  or
   die "$dateiname laesst sich nicht oeffnen\n"
print DATEI @zeilen  or
  die "$dateiname ist nicht schreibbar\n"
close DATEI  or  die "$dateiname ist nicht schreibbar\n"
-  SNIP 

And the python way looks like this:

-  SNIP 
#!/usr/bin/python
import sys

# Dateiname ermitteln
dateiname = sys.argv[ 1 ]

# Datei lesen
try:
datei = open( dateiname, 'r' )
zeilen = datei.readlines()
datei.close()
except IOError:
print dateiname, "ist nicht lesbar"
sys.exit( 1 )

# Datei sortieren
zeilen.sort()
dateiname = dateiname + '.sortiert'

# Datei schreiben
try:
datei = open( dateiname, 'w' )
datei.writelines( zeilen )
datei.close()
except IOError:
print dateiname, "ist nicht schreibbar"
-  SNIP 

Some other thing is that python knows about classes :-) But I do not
want to start a discussion about pro perl or python :-)

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//DtFvvmCkIIgH8QRAu8RAJoCCmI1b7l0I61JhxfITE7fs3UWNwCeKJJD
2VdRCbn8f8ABcuzSewy7L/8=
=Mg3n
-END PGP SIGNATURE-

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Hi all!

bid> Since debian-cd is no longer available to the public, it is very
bid> difficult to test currently. Maybe someone could provide a
bid> current .deb package for testing or unstable, so people can
bid> actually try out the most recent version.

But why isn't it? I used it for a long time to provide most recent
testing and unstable images of debian here - still the old one works,
but for how long? Is there any way to get access to the cvs version by
now?

bid> DVD worked fine for 3.0r1/i386 (large ISO9660), I don't know
bid> about other architectures and I could not check this for 3.0r2
bid> yet.

I got working DVD here too, and also for 3.0r2. By the way, are there
any "official" ones right now? Last year (long ago :-) we talked about
the images and resulting jigdo templates here - they were/are very
big..

bid> Are the "maintainers" of the "testing" and "unstable" CDs/DVD
bid> images still using debian-cd or something else ?

For my part I use debian-cd - like written above an old version, in
fact the last one I got from cvs before the compromise of debian's
servers. Since then, every time I tried to logon to cvs.d.o I got a
"connection refused" after logging in as an anonymous user.

>> - how do you think we should proceed ? should I start from a
>> completely new repository and import only what I want to keep ?
>> should we work in a branch of the actual repository ?

I would say, starting a new repository would be the best case, since I
think a (more or less) new debian-cd script will be in many cases
differernt form the actual one.

bid> For me, it was a bit difficult to follow all the source code and
bid> understand about including and excluding packages. Maybe a
bid> central documentation file for people who want to help modify or
bid> extend debian-cd would be great. While bash is not the best
bid> programming language, I can't think of a more suitable one; maybe
bid> Perl.

Thanks Bernd! That would be my idea - the old one is very bad
documentated right, and it took me a long time to understand what it
does. So I also agree to do it now in script language - perl or python
would be glad. If I had to decide I would prefer python, it's easier
to maintain and read for "beginners" (and I have done a lot of work in
python but just some snippets of perl in the past, so this is my point
of view).

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//BmSvvmCkIIgH8QRAiNgAJ9SoMNG/tuQdX/YdrQK/rVI3zqgXwCgwlEW
4KpZ46gjBqN589TLBU3ueas=
=tjpK
-END PGP SIGNATURE-

Re: Future of debian-cd

[Joey Hess]

Raphael Hertzog wrote:
> Here are the main critics that I have :

Let me add my main problem:

- Requires a full debian mirror to work.

IMHO, it would be much better to follow the lead of how d-i builds its
boot images, by using apt to download (u)debs. debian-cd would also need
to use wget or something for the non-deb components, but it should be
possible to use it with nothing but a network connection, and caching.

-- 
see shy jo


signature.asc
Description: Digital signature

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello,

bid> Well, I always had my doubts about Python. Of course, I have done
bid> next to nothing with Python and quite some code in Perl, so quite
bid> different experiences ;)

bid> Of course Perl has other problems like readability of code and
bid> soon compatibility problems when they switch from 5 to 6.

That's why I prefered python ;-) It's very easy to read and
understand: Look here... some thing I found on a website - it is in
german but I think it doesn't matter if I do not translate it:

Here is the perl way of opening a file, reading it and sorting the
lines into a new file:

-  SNIP 
#!/usr/bin/perl -w

# Dateiname ermitteln
$dateiname = shift;

# Datei lesen
open DATEI, "<$dateiname"  or
  die "$dateiname laesst sich nicht oeffnen\n"
@zeilen =   or
  die "$dateiname ist nicht lesbar\n"
close DATEI;

# sortieren
@zeilen = sort @zeilen;
$dateiname .= '.sortiert';

# Datei schreiben
open DATEI, ">$dateiname"  or
   die "$dateiname laesst sich nicht oeffnen\n"
print DATEI @zeilen  or
  die "$dateiname ist nicht schreibbar\n"
close DATEI  or  die "$dateiname ist nicht schreibbar\n"
-  SNIP 

And the python way looks like this:

-  SNIP 
#!/usr/bin/python
import sys

# Dateiname ermitteln
dateiname = sys.argv[ 1 ]

# Datei lesen
try:
datei = open( dateiname, 'r' )
zeilen = datei.readlines()
datei.close()
except IOError:
print dateiname, "ist nicht lesbar"
sys.exit( 1 )

# Datei sortieren
zeilen.sort()
dateiname = dateiname + '.sortiert'

# Datei schreiben
try:
datei = open( dateiname, 'w' )
datei.writelines( zeilen )
datei.close()
except IOError:
print dateiname, "ist nicht schreibbar"
-  SNIP 

Some other thing is that python knows about classes :-) But I do not
want to start a discussion about pro perl or python :-)

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//DtFvvmCkIIgH8QRAu8RAJoCCmI1b7l0I61JhxfITE7fs3UWNwCeKJJD
2VdRCbn8f8ABcuzSewy7L/8=
=Mg3n
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re[2]: Future of debian-cd

[Jan Kesten]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Hi all!

bid> Since debian-cd is no longer available to the public, it is very
bid> difficult to test currently. Maybe someone could provide a
bid> current .deb package for testing or unstable, so people can
bid> actually try out the most recent version.

But why isn't it? I used it for a long time to provide most recent
testing and unstable images of debian here - still the old one works,
but for how long? Is there any way to get access to the cvs version by
now?

bid> DVD worked fine for 3.0r1/i386 (large ISO9660), I don't know
bid> about other architectures and I could not check this for 3.0r2
bid> yet.

I got working DVD here too, and also for 3.0r2. By the way, are there
any "official" ones right now? Last year (long ago :-) we talked about
the images and resulting jigdo templates here - they were/are very
big..

bid> Are the "maintainers" of the "testing" and "unstable" CDs/DVD
bid> images still using debian-cd or something else ?

For my part I use debian-cd - like written above an old version, in
fact the last one I got from cvs before the compromise of debian's
servers. Since then, every time I tried to logon to cvs.d.o I got a
"connection refused" after logging in as an anonymous user.

>> - how do you think we should proceed ? should I start from a
>> completely new repository and import only what I want to keep ?
>> should we work in a branch of the actual repository ?

I would say, starting a new repository would be the best case, since I
think a (more or less) new debian-cd script will be in many cases
differernt form the actual one.

bid> For me, it was a bit difficult to follow all the source code and
bid> understand about including and excluding packages. Maybe a
bid> central documentation file for people who want to help modify or
bid> extend debian-cd would be great. While bash is not the best
bid> programming language, I can't think of a more suitable one; maybe
bid> Perl.

Thanks Bernd! That would be my idea - the old one is very bad
documentated right, and it took me a long time to understand what it
does. So I also agree to do it now in script language - perl or python
would be glad. If I had to decide I would prefer python, it's easier
to maintain and read for "beginners" (and I have done a lot of work in
python but just some snippets of perl in the past, so this is my point
of view).

Cheers,
Jan
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.2 (MingW32)
Comment: Benutzt PGP! Fragen? [EMAIL PROTECTED]

iD8DBQE//BmSvvmCkIIgH8QRAiNgAJ9SoMNG/tuQdX/YdrQK/rVI3zqgXwCgwlEW
4KpZ46gjBqN589TLBU3ueas=
=tjpK
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Future of debian-cd

[Joey Hess]

Raphael Hertzog wrote:
> Here are the main critics that I have :

Let me add my main problem:

- Requires a full debian mirror to work.

IMHO, it would be much better to follow the lead of how d-i builds its
boot images, by using apt to download (u)debs. debian-cd would also need
to use wget or something for the non-deb components, but it should be
possible to use it with nothing but a network connection, and caching.

-- 
see shy jo


signature.asc
Description: Digital signature

Re: Future of debian-cd

[Richard Atterer]

On Wed, Jan 07, 2004 at 02:16:39PM +0100, Raphael Hertzog wrote:
> - complicated shell code in Makefile is ugly, it was supposed to
>   not need to be modified, and in fact many parts of the Makefile
>   were modified for various purposes

I think the idea of using a Makefile at all wasn't that good in retrospect. 
Maybe it would be nicer to use "run-parts": A top-level script executes
everything in a "stages" directory in alphabetical order. The individual
stages could then have further directories of their own where
substage-scripts for that stage of the process are stored. This would get 
rid of all that "hook" ugliness...

There could be different top-level "stages" directories for the different 
CD flavours, with symlinks to scripts which are identical between any two 
of them. This has the advantage that you can fork code any time; instead of 
linking from the sarge version to the woody version, just make a copy of 
the script, then make changes/hacks which are woody-only.

Each top-level "stages" dir could also come with configuration files 
preconfigured for that particular CD flavour.

Well, something like that... I don't know if the above would work. :)

> - we have too many imbricated options which do not make sense to many
>   of us, it's time to see if we shouldn't get rid of some of them
>   (or at least hide them somewhere deeper than CONF.sh)

Yeah, CONF.sh has become far too big. It might make sense to split stuff up 
into several config files - after all, if e.g. you don't want to generate 
.jigdo files, you don't even need to look at the related options.

> - how do you think we should proceed ? should I start from a completely
>   new repository and import only what I want to keep ? should we work in
>   a branch of the actual repository ?

IMHO the kind soul who tackles this should start from scratch. :-7

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯

Re: Future of debian-cd

[Richard Atterer]

On Wed, Jan 07, 2004 at 02:16:39PM +0100, Raphael Hertzog wrote:
> - complicated shell code in Makefile is ugly, it was supposed to
>   not need to be modified, and in fact many parts of the Makefile
>   were modified for various purposes

I think the idea of using a Makefile at all wasn't that good in retrospect. 
Maybe it would be nicer to use "run-parts": A top-level script executes
everything in a "stages" directory in alphabetical order. The individual
stages could then have further directories of their own where
substage-scripts for that stage of the process are stored. This would get 
rid of all that "hook" ugliness...

There could be different top-level "stages" directories for the different 
CD flavours, with symlinks to scripts which are identical between any two 
of them. This has the advantage that you can fork code any time; instead of 
linking from the sarge version to the woody version, just make a copy of 
the script, then make changes/hacks which are woody-only.

Each top-level "stages" dir could also come with configuration files 
preconfigured for that particular CD flavour.

Well, something like that... I don't know if the above would work. :)

> - we have too many imbricated options which do not make sense to many
>   of us, it's time to see if we shouldn't get rid of some of them
>   (or at least hide them somewhere deeper than CONF.sh)

Yeah, CONF.sh has become far too big. It might make sense to split stuff up 
into several config files - after all, if e.g. you don't want to generate 
.jigdo files, you don't even need to look at the related options.

> - how do you think we should proceed ? should I start from a completely
>   new repository and import only what I want to keep ? should we work in
>   a branch of the actual repository ?

IMHO the kind soul who tackles this should start from scratch. :-7

Cheers,

  Richard

-- 
  __   _
  |_) /|  Richard Atterer |  GnuPG key:
  | \/¯|  http://atterer.net  |  0x888354F7
  ¯ '` ¯


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Re: Future of debian-cd

[bernd]

> Now, I'd like to hear from you :
> - what are your (constructive) critics against debian-cd ?

Since debian-cd is no longer available to the public, it is very difficult
to test currently. Maybe someone could provide a current .deb package for
testing or unstable, so people can actually try out the most recent version.

> - what design ideas do you have for a possible rewrite ?
> - what do you think of what i've written above ?

DVD worked fine for 3.0r1/i386 (large ISO9660), I don't know about other 
architectures and I could not check this for 3.0r2 yet.
Are the "maintainers" of the "testing" and "unstable" CDs/DVD images
still using debian-cd or something else ?

> - how do you think we should proceed ? should I start from a completely
>   new repository and import only what I want to keep ? should we work in
>   a branch of the actual repository ?

For me, it was a bit difficult to follow all the source code and understand
about including and excluding packages. Maybe a central documentation file
for people who want to help modify or extend debian-cd would be great.
While bash is not the best programming language, I can't think of a more
suitable one; maybe Perl.

Regards
Bernd Hentig

Future of debian-cd

[Raphael Hertzog]

Hello everyone,

many of you already noticed that debian-cd is a bit complicated.
And it tends to get worse since we keep doing small modifications
everywhere without a global logic.

Thus I think that debian-cd needs to be partially rewritten.

Here are the main critics that I have :
- complicated shell code in Makefile is ugly, it was supposed to
  not need to be modified, and in fact many parts of the Makefile
  were modified for various purposes
- you can't reasonnably build potato cd with current debian-cd because
  of those "infrastructural change" to a CD (inclusion of udeb for
  example) which were not expected at the time of my initial rewrite
  of debian-cd. I still managed to keep most of the compatibility
  but at the cost of more code complexity.
- we have too many imbricated options which do not make sense to many
  of us, it's time to see if we shouldn't get rid of some of them
  (or at least hide them somewhere deeper than CONF.sh)
- the official support of businesscard cd/netinst cd/dvd is not good, most
  people are not able to build such CD with debian-cd because they don't
  know how the magic combination of parameters.
- it's too much of black magick to know the good size parameters to give

In a new design, I would like to have :
- a MEDIA_TYPE input variable that would tell me :
  1 - businesscard CD
  2 - netinst CD
  3 - usual CD
  4 - DVD
  The size would be taken from a default value (if not overriden).
- a system of profile that would describe how to build a particular kind
  of ISO
- a system of profile for debian-installer, so that we can easily
  customize the udebs used by the debian installer

The new design should still reuse most of the existing code, not
everything needs to be thrown. In fact I think that most should be kept
but cleaned from the cruft and reorganized in a different way.

Now, I'd like to hear from you :
- what are your (constructive) critics against debian-cd ?
- what design ideas do you have for a possible rewrite ?
- what do you think of what i've written above ?
- how do you think we should proceed ? should I start from a completely
  new repository and import only what I want to keep ? should we work in
  a branch of the actual repository ?

BTW, as you may have noticed I have less & less time for debian-cd, and
I'd like to have some help in the long term to help me maintain it. With
a (partial) rewrite, it would be the perfect time for someone to step up
and get familiar with the code ...

Cheers,
-- 
Raphaël Hertzog -+- http://www.ouaza.com
Formation Linux et logiciel libre : http://www.logidee.com
Earn money with free software: http://www.geniustrader.org

Re: Future of debian-cd

[bernd]

> Now, I'd like to hear from you :
> - what are your (constructive) critics against debian-cd ?

Since debian-cd is no longer available to the public, it is very difficult
to test currently. Maybe someone could provide a current .deb package for
testing or unstable, so people can actually try out the most recent version.

> - what design ideas do you have for a possible rewrite ?
> - what do you think of what i've written above ?

DVD worked fine for 3.0r1/i386 (large ISO9660), I don't know about other 
architectures and I could not check this for 3.0r2 yet.
Are the "maintainers" of the "testing" and "unstable" CDs/DVD images
still using debian-cd or something else ?

> - how do you think we should proceed ? should I start from a completely
>   new repository and import only what I want to keep ? should we work in
>   a branch of the actual repository ?

For me, it was a bit difficult to follow all the source code and understand
about including and excluding packages. Maybe a central documentation file
for people who want to help modify or extend debian-cd would be great.
While bash is not the best programming language, I can't think of a more
suitable one; maybe Perl.

Regards
Bernd Hentig


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]

Future of debian-cd

[Raphael Hertzog]

Hello everyone,

many of you already noticed that debian-cd is a bit complicated.
And it tends to get worse since we keep doing small modifications
everywhere without a global logic.

Thus I think that debian-cd needs to be partially rewritten.

Here are the main critics that I have :
- complicated shell code in Makefile is ugly, it was supposed to
  not need to be modified, and in fact many parts of the Makefile
  were modified for various purposes
- you can't reasonnably build potato cd with current debian-cd because
  of those "infrastructural change" to a CD (inclusion of udeb for
  example) which were not expected at the time of my initial rewrite
  of debian-cd. I still managed to keep most of the compatibility
  but at the cost of more code complexity.
- we have too many imbricated options which do not make sense to many
  of us, it's time to see if we shouldn't get rid of some of them
  (or at least hide them somewhere deeper than CONF.sh)
- the official support of businesscard cd/netinst cd/dvd is not good, most
  people are not able to build such CD with debian-cd because they don't
  know how the magic combination of parameters.
- it's too much of black magick to know the good size parameters to give

In a new design, I would like to have :
- a MEDIA_TYPE input variable that would tell me :
  1 - businesscard CD
  2 - netinst CD
  3 - usual CD
  4 - DVD
  The size would be taken from a default value (if not overriden).
- a system of profile that would describe how to build a particular kind
  of ISO
- a system of profile for debian-installer, so that we can easily
  customize the udebs used by the debian installer

The new design should still reuse most of the existing code, not
everything needs to be thrown. In fact I think that most should be kept
but cleaned from the cruft and reorganized in a different way.

Now, I'd like to hear from you :
- what are your (constructive) critics against debian-cd ?
- what design ideas do you have for a possible rewrite ?
- what do you think of what i've written above ?
- how do you think we should proceed ? should I start from a completely
  new repository and import only what I want to keep ? should we work in
  a branch of the actual repository ?

BTW, as you may have noticed I have less & less time for debian-cd, and
I'd like to have some help in the long term to help me maintain it. With
a (partial) rewrite, it would be the perfect time for someone to step up
and get familiar with the code ...

Cheers,
-- 
Raphaël Hertzog -+- http://www.ouaza.com
Formation Linux et logiciel libre : http://www.logidee.com
Earn money with free software: http://www.geniustrader.org


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]