Bug#1076199: www.debian.org: Webpage Debian-Installer add riscv64 other images link

[Cyril Brulebois]

Colin Watson  (2024-07-12):
> I suspect this is mostly because debian-installer riscv64 support hasn't
> been uploaded to unstable yet, so it isn't on
> https://deb.debian.org/debian/dists/testing/main/installer-riscv64/ or
> https://deb.debian.org/debian/dists/unstable/main/installer-riscv64/.
> I'm not sure what the plan for that is; somebody on debian-boot@
> probably knows.
> 
> Once that's done, riscv64 will need to be added to the
> devel-images-arches tag (and I suppose also trixie-images-arches) in
> webwml:english/template/debian/installer.wml.

That'll get sorted out when a release is published, but thanks for
mentioning it.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#851541: Bug#902668: Draft for rewrite of https://www.debian.org/CD/verify

[Cyril Brulebois]

Hi,

Tassia Camoes Araujo  (2024-03-25):
> I've reviewed the proposed patch, and I think it should be applied as
> soon as possible.
> 
> It seems Laura was waiting for a final review before applying this patch
> (long overdue!), which IMHO would bring much more clarity to the image
> verification process (usually, a big struggle to new users).
> 
> We should make a decision about the long key IDs request (points 1 and 2
> from #851541), and once those changes go online, I think both bugs could
> be closed (#902668 and #851541).
> 
> Thanks for all who have invested energy to clarify this process, and I
> hope we can benefit from your work very soon!
> 
> Cheers,
> 
> Tassia. 

Cc += debian-cd@ for information.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: new global .transignore file for webwml repo

[Cyril Brulebois]

Cyril Brulebois  (2023-10-02):
> > MailingLists/subscribe.wml
> > MailingLists/unsubscribe.wml
> 
> NACK.

Ah, based on Paulo's answer I'll clarify: (un)subscription information
must be available in as many languages as possible in my opinion.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: new global .transignore file for webwml repo

[Cyril Brulebois]

Thomas Lange  (2023-10-01):
> I've already asked on the #debian-www channel for comments but I got
> only the question if the regex are anchored. If I get no objection, I
> like to install this .transignore file next week.
> 
> 
> # here you can define global Perl regex to match file that are excluded
> News/\d{4}

NACK.

> News/weekly/\d{4}

NACK.

> MailingLists/subscribe.wml
> MailingLists/unsubscribe.wml

NACK.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: webwml failed

[Cyril Brulebois]

Debian Webmaster  (2023-09-28):
> /srv/www.debian.org/cron/log/wml_run.log:ePerl:Error: Perl runtime error 
> (interpreter rc=0)
> /srv/www.debian.org/cron/log/wml_run.log-
> /srv/www.debian.org/cron/log/wml_run.log- Contents of STDERR channel: 
> -
> /srv/www.debian.org/cron/log/wml_run.log:count_changes() ERROR: commit rev1 
> 18c4b33fe6dcee4c5cb83603609a9b4bd481da0c  not found in revisions of 
> ../../..//english/CD/live/index.wml
> /srv/www.debian.org/cron/log/wml_run.log---
> /srv/www.debian.org/cron/log/wml_run.log:** WML:Break: Error in Pass 3 (rc=1).

Fixed: 
https://salsa.debian.org/webmaster-team/webwml/-/commit/b1ab0700ee535809875d557458de06536c7ac699


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1050675: www.debian.org: French bookworm d-i web page jumped the shark

[Cyril Brulebois]

Package: www.debian.org
Severity: important
Tags: l10n
X-Debbugs-Cc: debian-l10n-fre...@lists.debian.org

Hi,

https://www.debian.org/releases/bookworm/debian-installer/index.fr.html
has an interesting view about the state of the Debian releases, claiming
Debian 12 has been overthrown by Debian 13, and recommending to install
Trixie instead.

Other translations look good to me, so I don't think this is a general
problem with a macro, but something really bad at the individual
translation level.

I haven't investigated the wml source though; French team in copy.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#1037490: www.debian.org: clean up old files

[Cyril Brulebois]

(Cc-ing Adam for DSA following the ping about disk space earlier.)

Cyril Brulebois  (2023-06-13):
> Spotted while working on #1037479: we have a bunch of files around that
> are no longer useful, because relevant suites are EOL and archived, and
> have been dropped from the relevant config files, templates, etc.
> 
> They could probably be cleaned up, but I don't want to rush this kind of
> things, hence this bug report.

Apparently we're going to have to do something soon, since the recent
updates in picconi have led to a noticeable and worrisome bump in
storage used:

/dev/mapper/vg0-srv  246G  234G  822M  100%  /srv

Some cleanup happened in user directories, but I suppose it would make
sense to keep track of our disk space usage:

/dev/mapper/vg0-srv  246G  136G  99G58%  /srv

And beware, there are peaks during runs, with e.g. tmp/ filing up.

Also, the mirror (pkgmirror-csail) should like this as well given:

/dev/vdb 148G  129G  12G92%  /srv

See also graphs (with the usual guest authentication):
  https://munin.debian.org/debian.org/picconi.debian.org/df.html
  https://munin.debian.org/debian.org/pkgmirror-csail.debian.org/df.html


Our directory looked like this when I started:

426M./archive
200K./bin
268K./cache
8.0K./cgi-bin
29M ./conf
64K ./cron.d
8.0K./debian
116G./files
16M ./.git
133M./home
300K./lib
14M ./mail
8.4G./mirror
4.0K./po
196K./static
116K./templates
7.5G./tmp
2.6G./www
---
135Gtotal

I was thinking of getting rid of these:

$ du -sch $(find -name 'jessie*')
65M ./mirror/202306141247/www/jessie-backports
17M ./mirror/202306141247/www/source/jessie-backports
87M ./mirror/202306141247/www/source/jessie
3.5M./mirror/202306141247/www/source/jessie-updates
5.3M./mirror/202306141247/www/source/jessie-backports-sloppy
505M./mirror/202306141247/www/jessie
12M ./mirror/202306141247/www/jessie-updates
6.4M./mirror/202306141247/www/jessie-backports-sloppy
196K./archive/backports/jessie-backports
26M ./www/jessie-backports
5.8M./www/source/jessie-backports
11M ./www/source/jessie
2.3M./www/source/jessie-updates
3.4M./www/source/jessie-backports-sloppy
176M./www/jessie
3.1M./www/jessie-updates
4.0M./www/jessie-backports-sloppy

930Mtotal

and these:

$ du -sch $(find -name 'stretch*')
85M ./mirror/202306141247/www/source/stretch
5.1M./mirror/202306141247/www/source/stretch-backports-sloppy
15M ./mirror/202306141247/www/source/stretch-backports
3.4M./mirror/202306141247/www/source/stretch-updates
416M./mirror/202306141247/www/stretch
6.2M./mirror/202306141247/www/stretch-backports-sloppy
52M ./mirror/202306141247/www/stretch-backports
4.9M./mirror/202306141247/www/stretch-updates
11M ./www/source/stretch
3.4M./www/source/stretch-backports-sloppy
5.8M./www/source/stretch-backports
2.2M./www/source/stretch-updates
179M./www/stretch
3.9M./www/stretch-backports-sloppy
22M ./www/stretch-backports
2.6M./www/stretch-updates
-
813Mtotal

But those easy ones would only account for 1.7G combined, as mentioned
yesterday.

Searching for files containing but not starting with either jessie or
stretch, there are a lot of other things that looked like they could go
away:
 - filelists_*
 - filenames_*
 - new_package_info_*
 - package_names_*
 - packages_all_*
 - reverse_*
 - source_names_*
 - sources_all_*

and I expected no collateral damages.

To be on the safe side, I created two tarballs, one for jessie, one for
stretch, containing all those files, before removing them. I've compressed
them so that they take less space than actual files, moved them in a
directory that shouldn't be mirrored to pkgmirror-csail, and I've
scheduled their deletions in 6 months (via at), which should leave plenty
of time to restore them if needed, while making sure full cleanup happens
eventually.

pkg_user@picconi:/srv/packages.debian.org$ echo 'rm -rv 
/srv/packages.debian.org/obsolete/' | at 'now + 6 months'
warning: commands will be executed using /bin/sh
job 5 at Thu Dec 14 18:56:00 2023
pkg_user@picconi:/srv/packages.debian.org$ atq
5   Thu Dec 14 18:56:00 2023 a pkg_user

Also removed, a long obsolete file:

-rw--- 1 pkg_user pkg_maint 7.5G May  2  2021 tmp/sort01fAge

Not sure why this file stays behind, but at least the name is stable, so
the non-freed space is getting reclaimed when the next run starts:

-rw-r--r-- 1 pkg_user pkg_maint 1479457587

Re: ReleaseCheckList wiki page for web team

[Cyril Brulebois]

Holger Wansing  (2023-06-14):
> I have added an entry to update packages.debian.org for the search
> interface (update mapping of codenames to stable|testing|unstable;
> much hardcoding of codenames there:-( )

Yes, this is… painful.

> Feel free to remove or change that, if you think it should be listed
> at a different place or similar (it's also listed on release-managers
> list btw)

That was a nice first draft, and something I was meaning to add today
(after fixing the biggest bugs in there), thanks for doing so.

I've switched that to:

See 
https://salsa.debian.org/webmaster-team/packages/-/compare/6bdfeb02f5...bde18c3458
for an example + plus deploy on picconi).

I'm not sure whether to push the local branch I worked on (and merged
into debian-master), which is an explicit `bookworm-is-released`, so
that it can easily be found when it's time to `trixie-is-released`.

In any case, that series of commits is a split of the big commit you
linked to into smaller chunks so that it can be done and reviewed
incrementally, hence the range of commits above. I suppose the
presence/absence of a specifically-named branch is a matter of taste, as
long as there's a reference to the actual commits in the wiki page… ;)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#992258: packages.debian.org still not showing bullseye-security packages

[Cyril Brulebois]

Cyril Brulebois  (2023-06-13):
> Let's see if those help:
>   
> https://salsa.debian.org/webmaster-team/packages/-/compare/f673a47d01...4111b73ede

That wasn't sufficient, because of some earlier check/optimization. And
there was another thing that was missed, which would have prevented it
from working anyway.

New attempt:
  
https://salsa.debian.org/webmaster-team/packages/-/compare/cf3a6bc92c...4894715a8a

> As usual, I'm not forcing a refresh right away, and I'll let the next
> mirror pulse trigger reindexing. We should have a better idea in 6-12
> hours.

Still valid.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#992258: packages.debian.org still not showing bullseye-security packages

[Cyril Brulebois]

(Adding back submitter, and everyone who mailed that bug report.)

Hi,

Colomban Wendling  (2022-10-25):
> I have unfortunately no insight on how this could be fixed, but it's
> rather inconvenient as packages.debian.org is thus effectively not a
> trustable source for Bullseye.

Let's see if those help:
  
https://salsa.debian.org/webmaster-team/packages/-/compare/f673a47d01...4111b73ede

As usual, I'm not forcing a refresh right away, and I'll let the next
mirror pulse trigger reindexing. We should have a better idea in 6-12
hours.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1037490: www.debian.org: clean up old files

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org

Hi,

Spotted while working on #1037479: we have a bunch of files around that
are no longer useful, because relevant suites are EOL and archived, and
have been dropped from the relevant config files, templates, etc.

They could probably be cleaned up, but I don't want to rush this kind of
things, hence this bug report.

Trivial candidates:

kibi@picconi:/srv/packages.debian.org$ du -sch 
www/{,source/}{jessie,stretch}*
681Mwww/jessie
91M www/jessie-backports
11M www/jessie-backports-sloppy
15M www/jessie-updates
594Mwww/stretch
73M www/stretch-backports
9.9Mwww/stretch-backports-sloppy
7.3Mwww/stretch-updates
97M www/source/jessie
22M www/source/jessie-backports
8.5Mwww/source/jessie-backports-sloppy
5.7Mwww/source/jessie-updates
95M www/source/stretch
21M www/source/stretch-backports
8.3Mwww/source/stretch-backports-sloppy
5.5Mwww/source/stretch-updates
1.7Gtotal


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#1037488: www.debian.org: drop static/packages-site.css entirely?

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org

Hi,

Spotted while working on #1037479, static/packages-site.css hardcodes
suite names. But digging a little deeper, it doesn't seem to be imported
anymore, after the huge redesign:

commit 74376d3a2530091edba43c51241022ca26977b7b
Date:   Thu Oct 28 16:26:15 2010 +0200

Probably best to finally drop it, to avoid wasting more brain time on
suite updates (for that particular file)?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#1037479: 500 Internal Server Error accessing https://packages.debian.org/testing/name_of_package

[Cyril Brulebois]

Control: tag -1 patch pending

Hi,

And yes, thanks for reporting!

Laura Arjona Reina  (2023-06-13):
> Thanks for the report. It seems that our website packages.debian.org
> still doesn't recognize trixie being testing.

It's worse than that: it doesn't know about trixie at all yet. There are
many places were suites are hardcoded, and I missed adding trixie to one
of the central place (config.sh.sed.in). That's fixed now, and it should
be better after the next mirror push (which triggers indexing). Note the
previous push/indexing is still running, and I really don't want to try
and add a manual run in there, so waiting something like 6 to 12 hours
should be our safest bet.

https://salsa.debian.org/webmaster-team/packages/-/commit/bde18c345814d497ddd0537471759a2d9aad2fcd

Tagging accordingly, the first person to spot the right files popping up
in the right places can close this bug report.


While looking around I spotted the CSS also hardcode suite names. I'll
look into that next. Also spotted leftover files from old suites
(jessie, stretch), will file a different ticket for those.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: page DebianUpgrade suggests "bookworm stable" which looks wrong

[Cyril Brulebois]

Hi Andrew,

Andrew E Parso-York  (2023-05-25):
> Apology for not using reportbug, but this is a simple wiki text error and
> I'm not confident that my proposed change is correct.
> 
> https://wiki.debian.org/DebianUpgrade contains the following:
> 
> # If you are migrating to Bookworm or later, then a new repo for non-free
> firmware is available.
> # If you wish, you can add non-free and non-free-firmware, depending on your
> specific needs.
> # For instance, the line
> #    deb https://deb.debian.org/debian/ bookworm stable main contrib
> non-free non-free-firmware
> 
> I don't think "stable" belongs there following bookworm.
> 
> It's either bookworm, OR a future release, OR stable, not both.
> 
> Thank you, and sorry I couldn't figure out how to use reportbug.

You're absolutely correct, feel free to remove “ stable” from there.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1035960: All of sudden, the Spanish PO debconf templates is getting full of alien translators :-)

[Cyril Brulebois]

Cyril Brulebois  (2023-05-12):
> I'll keeping looking at what's supposed to happen on tye, but I'm not
> sure I'll be able to get to the bottom of it on my own.

At least there's a HUGE red flag on tye. Load to the roof, RAM/swap
almost full, lots of dl10n-spider processes running for the same
language, some of them started May 9th.

kibi@tye:~$ uptime
 10:02:58 up 12 days, 21:47,  2 users,  load average: 63.24, 64.57, 66.51

kibi@tye:~$ free -h
   totalusedfree  shared  buff/cache   
available
Mem:   1.9Gi   1.7Gi69Mi   1.0Mi   125Mi
57Mi
Swap:  511Mi   511Mi   0.0Ki

kibi@tye:~$ ps faux|grep dl10n-spider|grep -o -- '--check-bts ..'|sort|uniq 
-c
  4 --check-bts ca
  1 --check-bts cs
  1 --check-bts da
 51 --check-bts de
  7 --check-bts es
  2 --check-bts fr

kibi@tye:~$ ps faux|awk '/CRON/ {print $9}'|sort|uniq -c
 11 May09
 23 May10
 23 May11
  1 00:15
  1 02:15
  1 03:15
  1 04:15
  1 05:15
  1 06:15
  1 07:15
  1 08:13
  1 08:15
  1 09:15
  2 10:00
  1 10:01

Note that many de.po occurrences appear in the status file for other
languages, looks like processes heavily stomping onto others' feet?


It looks to me there should be some locking at the very least to avoid
that amount of concurrency. And that it would probably be best to start
afresh, killing all those processes, maybe disabling the cron jobs,
cleaning temporary and maybe corrupted data files, and triggering a
single run manually to see if it works.

But then, I have 0 knowledge about the spider, and I'll leave that up to
someone else: I don't want to risk making the matter worse!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1035960: All of sudden, the Spanish PO debconf templates is getting full of alien translators :-)

[Cyril Brulebois]

Hi,

Laura Arjona Reina  (2023-05-12):
> Maybe this issue in the Spanish dashboards is related to the report
> #1035960 about issues in the German dashboards:
> 
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1035960
> 
> I will try to investigate both issues but any help is welcome.
> 
> If any of you is sure about the last date when you saw things were
> working well, please tell, so I can try to learn what happened since
> then that could cause the issue.

Please pretend I don't know anything about i18/l10n, www, or how the
latter uses information published by the former…

It looks to me like at least the status.$LANG files exposed at [1] are
busted, with many references to different $LANG.po in some files.

 1. https://l10n.debian.org/coordination/00data/

For example, status.es has 5319 references to de.po…


Whether that's linked or not, I didn't find reassuring that the www
build output hundreds/thousands of warnings when running a local,
partial build of english/international…

I'll keeping looking at what's supposed to happen on tye, but I'm not
sure I'll be able to get to the bottom of it on my own.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: debian download link broken

[Cyril Brulebois]

Hi Tom,

Tom Owad  (2023-04-29):
> The main download link on https://www.debian.org/download is broken.
> It goes to debian-11.6.0-amd64-netinst.iso, but that's removed and
> 11.7 is up now.

Thanks for letting us know. I think the website update for the point
release is or was ready in time but we might have had some slight
hiccup synchronization-wise once the cdimage building finished.

I'd expect that to be fixed in the next few hours, and as you noted,
hopefully people can navigate their way on cdimage to either get
11.6.0 in the archive directory or 11.7.0 in the canonical location
until then.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Debian testing non-free firmware packages are not available

[Cyril Brulebois]

Hi Piotr,

Quoting Andrew's answer in full:

Andrew M.A. Cater  (2023-02-11):
> On Sat, Feb 11, 2023 at 05:22:24PM +, Piotr Lopatka wrote:
> > Hi Debian maintainer,
> > 
> > Just reporting that the website seems down as non-free packages are not
> > available for install/update. It looks like testing and Sid are affected.
> > Debian stable seems fine.
> > 
> 
> Have you added non-free-firmware to the /etc/apt/sources.list lines?
> 
> https://lists.debian.org/debian-user/2023/01/msg00706.html gives information.
> 
> All the very best,
> 
> Andy Cater

From your initial message, it's not clear whether you were reporting
problems on the APT side, or on the website. If that's about the latter,
packages.debian.org should know about packages in the non-free-firmware
component now.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Debian > firmware-linux

[Cyril Brulebois]

Hallo wieder,

Cyril Brulebois  (2023-02-12):
> Yeah, given the usual lot of uncommitted changes on picconi (the machine
> acting as packages.d.o master) I initially decided against trying to add
> non-free-firmware support there.
> 
> Not having those packages listed really doesn't help users (or even
> developers) understand what's going on and where packages went, so I've
> just live-patched a few files, which should help get things going. It
> might take a few hours for the next update to hit the new config though.

I've hotpatched another file, and I'm seeing some results. Please give
it until tomorrow, and feel free to follow up if you still spot missing
packages.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: missing non-free-firmware support

[Cyril Brulebois]

Fixed.
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: missing non-free-firmware support

[Cyril Brulebois]

Known.

(See? Can be terse too.)
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Debian > firmware-linux

[Cyril Brulebois]

Hallo Thilo,

Thilo Six  (2023-02-12):
> as it seems packages.debian.org does not reflect this change either:
> https://www.debian.org/releases/bookworm/amd64/release-notes/ch-whats-new.en.html#archive-areas
> 
> packages.debian.org misses non-free-firmware

Yeah, given the usual lot of uncommitted changes on picconi (the machine
acting as packages.d.o master) I initially decided against trying to add
non-free-firmware support there.

Not having those packages listed really doesn't help users (or even
developers) understand what's going on and where packages went, so I've
just live-patched a few files, which should help get things going. It
might take a few hours for the next update to hit the new config though.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1030009: www.debian.org: addition of non-free-firmware for bookworm and higher

[Cyril Brulebois]

cc += debian-cd

Holger Wansing  (2023-02-11):
> I have already updated several of files you listed (those which changings
> were most likely non-debatable), but now I would like to discuss, what to
> do with files like
> 
> releases/bullseye/debian-installer/index.wml
> releases/bullseye/errata.wml
> releases/buster/debian-installer/index.wml
> releases/buster/errata.wml
> 
> Those old release pages have a warning hint like
> 
> If any of the hardware in your system requires non-free firmware to be 
> loaded with the device driver, you can use one of the tarballs of common 
> firmware packages or download an unofficial image including these non-free 
> firmwares. Instructions how to use the tarballs and general information 
> about loading firmware during an installation can be found in the 
> Installation Guide.
> 
> unofficial images with firmware included - daily builds
> 
> unofficial images with firmware included - weekly builds
> 
> 
> First I thought we can leave this completely untouched, since these
> "images with firmware included" are still there and they are also still
> useful (for some hardware at least), if one wants to install those old 
> releases. 
> But now I would like to propose, we change the "unofficial images with 
> firmware included" phrase, since they are no longer *unofficial* according
> to the non-free-firmware GR.
> Thus I propose to change this into "special images with firmware included"
> for those old releases.
> 
> Comments?

I don't think we should change anything for buster and bullseye. We
aren't going to produce official images with main and non-free-firmware,
and I don't think the current implementation of those unofficial builds
(using non-free) would match the spirit of the GR.

Maybe people feel differently, but the guiding principle behind all the
changes I've driven so far is: the official status is for bookworm and
later (same story as for packages in non-free-firmware, even if that
component popped up in earlier suites in the archive).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1030009: www.debian.org: addition of non-free-firmware for bookworm and higher

[Cyril Brulebois]

Cyril Brulebois  (2023-01-30):
> I'm also trying to get documentation ready, and you can read more about
> it in two merge requests (freshly opened, not reviewed yet):
>  - 
> https://salsa.debian.org/installer-team/installation-guide/-/merge_requests/23
>  - https://salsa.debian.org/ddp-team/release-notes/-/merge_requests/138

Extra modification on the cdimage.debian.org side (that was alluded to
in the installation guide merge request), firmware material is now
available from:
  https://cdimage.debian.org/cdimage/firmware/

in addition to the old location:
  https://cdimage.debian.org/cdimage/unofficial/non-free/firmware/

Server-side, there's a symlink, meaning both locations serve the same
files, e.g. for all distributions (i.e. not just bookworm and above in
the new location).

That means there's no absolute need to update that URL, but it would
be nicer to do so. :)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1030009: www.debian.org: addition of non-free-firmware for bookworm and higher

[Cyril Brulebois]

Package: www.debian.org
Severity: important

Hi,

This was mentioned a few days ago on debian-project@ by Gunnar:
  https://lists.debian.org/debian-project/2023/01/msg00018.html

but much work has happened since then, with debian-installer components
having been extended to leverage this new component. The next release
of the installer should happen within the next two weeks.

I'm also trying to get documentation ready, and you can read more about
it in two merge requests (freshly opened, not reviewed yet):
 - 
https://salsa.debian.org/installer-team/installation-guide/-/merge_requests/23
 - https://salsa.debian.org/ddp-team/release-notes/-/merge_requests/138

Unless you fancy reviewing what changed in the installer, I'd skip the
former and concentrate on the latter, which briefly explains why
non-free-firmware was added.

It's available for bookworm suites and above (might be for other suites,
but I don't expect packages to show there).

TL;DR:

 - bullseye systems are likely to have a sources.list configured this
   way:

 deb http://deb.debian.org/debian bullseye main contrib non-free

 - bullseye systems getting upgraded to bookworm should be modified with
   the following configuration (to make sure non-free firmware packages
   that might be installed have a chance to get upgraded):

 deb http://deb.debian.org/debian bookworm main contrib non-free 
non-free-firmware

 - new bookworm systems are likely to be configured with either of
   these (unless one picks an expert install, enabling contrib and/or
   non-free explicitly):

 deb http://deb.debian.org/debian bookworm main 
 deb http://deb.debian.org/debian bookworm main non-free-firmware


There's also some related ongoing work, about adding some hints in apt
(linking back to the release notes), and about fixing command-not-found
in bullseye:
 - https://salsa.debian.org/apt-team/apt/-/merge_requests/282
 - https://bugs.debian.org/1029803


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#855695: packages.debian.org shows non-existing binutils packages on arm64

[Cyril Brulebois]

Hi Jochen,

(Answering with my “I have the gid by accident” hat.)

Jochen Sprickerhof  (2023-01-20):
> There are more old files that should probably go:
> 
> jspricke@picconi:/srv/packages.debian.org/archive/debports$ find -type f 
> -mtime +0 -exec ls -l {} \;
> -rw-r--r-- 1 pkg_user pkg_maint 1029950 Nov  6  2014 
> ./experimental/Contents-arm64.gz
> -rw-r--r-- 1 pkg_user pkg_maint 1817658 Mai 19  2019 
> ./experimental/main/binary-powerpcspe/Packages.gz
> -rw-r--r-- 1 pkg_user pkg_maint 9198 Mai 19  2019 
> ./experimental/main/debian-installer/binary-powerpcspe/Packages.gz
> -rw-r--r-- 1 pkg_user pkg_maint 20 Nov  6  2014 
> ./experimental/main/debian-installer/binary-arm64/Packages.gz
> -rw-r--r-- 1 pkg_user pkg_maint 316060 Nov  6  2014 
> ./experimental/main/binary-arm64/Packages.gz
> -rw-r--r-- 1 pkg_user pkg_maint 5221435 Mai 19  2019 
> ./experimental/Contents-powerpcspe.gz
> -rw-r--r-- 1 pkg_user pkg_maint 27303127 Nov  6  2014 ./sid/Contents-arm64.gz
> -rw-r--r-- 1 pkg_user pkg_maint 95517 Mai 19  2019 
> ./sid/main/debian-installer/binary-powerpcspe/Packages.gz
> -rw-r--r-- 1 pkg_user pkg_maint 98536 Nov  6  2014 
> ./sid/main/debian-installer/binary-arm64/Packages.gz
> -rw-r--r-- 1 pkg_user pkg_maint 38478639 Mai 19  2019 
> ./sid/Contents-powerpcspe.gz

Looks good to me, all gone.

> There is also arm, avr32, mips, powerpcspe, s390 and sparc listed in
> /srv/packages.debian.org/config.sh. I guess they can be removed as well.

Given the git checkout is in the following state, I don't think it's
reasonable for me to touch anything else:

pkg_user@picconi:/srv/packages.debian.org$ git diff --stat|tail -1
     52 files changed, 12833 insertions(+), 10634 deletions(-)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1029265: www.debian.org: "Bugs/server-control" missing information about "Control:"

[Cyril Brulebois]

Hi Christian,

While improving the current doc can always happen, I'm not sure I agree
with your assessment that critical information is missing…

Christian Buhtz  (2023-01-20):
> https://www.debian.org/Bugs/server-control […]

The very first line mentions writing those mails to cont...@bugs.debian.org.

(That's the historical way of manipulating bug reports; support for
Control: when mailing n...@bugs.debian.org came much later.)

https://www.debian.org/Bugs/server-refcard is also very clear about
which commands can be sent to request@ and to control@.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1029012: www.debian.org: https://www.debian.org/releases/bullseye/debian-installer shows "trixie" information

[Cyril Brulebois]

Cyril Brulebois  (2023-01-16):
> I can imagine two scenarios:
>  - sync issue;
>  - web server issue.

Actually, even if files seem correct, some fixup in the webwml source
seems required:
 - 
https://salsa.debian.org/webmaster-team/webwml/-/commit/1e1313a4e274d3ef820eec07f239bcb43d9f42c4
 - 
https://salsa.debian.org/webmaster-team/webwml/-/commit/6d4f70c75dfdfa3c2734e836e5ce3d04d6bc6cb0

We might need to force a rebuild of the bullseye one, but I'm not sure
how to do that without generating work for translators; so I'll let
someone else from the website team pick up that topic. (Maybe fixing
paths will get the right files into place for bullseye, but that would
be luck in my opinion.)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1029012: www.debian.org: https://www.debian.org/releases/bullseye/debian-installer shows "trixie" information

[Cyril Brulebois]

Micah Anderson  (2023-01-16):
> english/releases/bullseye/debian-installer/index.wml doesn't seem crazy at 
> first glance…
> interestingly, grep -i trixie 
> /srv/www.debian.org/webwml/english/releases/bullseye/debian-installer/index.en.html
>  returns nothing.
> (so the build results seem to match the source files, which are fine; not 
> sure where the buggy served page comes from…)

wolkenstein seems to have correct files for bullseye and trixie (even if
that one doesn't make sense at this point); the bullseye link[1] clearly
leads to the trixie page's getting served (as confirmed via build
timestamps); the equivalent trixie link[2] 404's even though the file is
present on disk (again, at least on wolkenstein).

 1. https://www.debian.org/releases/bullseye/debian-installer/index.en.html
 2. https://www.debian.org/releases/trixie/debian-installer/index.en.html

Both of those seem fine though:
 - https://www.debian.org/releases/bullseye/
 - https://www.debian.org/releases/trixie/

I haven't spotted anything suspicious in dsa-puppet's
modules/roles/templates/apache-www.debian.org.erb that controls the web
serve.

I can imagine two scenarios:
 - sync issue;
 - web server issue.

The former doesn't seem very plausible: why would the trixie page have
been published instead of the bullseye one, and why would such a
possible mistake not have been fixed since then? (Again, *.en.html files
are correct on wolkenstein.)

The latter seems more plausible to me, even if I'm not sure why that's
happening. I suppose we would need help from DSA to figure out what's
going on on the web server (files actually present on the filesystem
plus which files are getting served and how).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1015172: marked as done (wiki.debian.org: Sid installation fails - No kernel modules found - Wiki issue)

[Cyril Brulebois]

Piscium  (2022-07-17):
> The Bullseye file that I used to successfully install Sid [3] is
> timestamped 05-Jul-2022 15:57, which is only 12 days old. If you
> compare the two links below in [2] and [3], the difference is that [2]
> is in unstable (one year old file) and [3] is in bullseye (12 days
> old). The bullseye file is recent whereas the Sid file is ancient.
> This is counter-intuitive.

We shouldn't have let so much time go by without putting out an alpha
release, so yes, the situation you're describing is less than ideal.

The debian-installer package got updated for the 11.4 point release
(bullseye), and it was propagated upward (prop-up) to testing/unstable
to satisfy version constraints; but the same doesn't happen for the
contents of the installer-* directories, hence the timestamp differences
you're seeing.

> Cyril said "there seem to be some weird things going on, mixing and
> matching bits from bullseye and unstable?". Yes, it seems so, or at
> least something along those lines.
> 
> [1] https://wiki.debian.org/DebianUnstable
> [2] 
> http://ftp.uk.debian.org/debian/dists/unstable/main/installer-amd64/current/images/netboot/mini.iso
> (FAILURE)
> [3] 
> http://ftp.uk.debian.org/debian/dists/bullseye/main/installer-amd64/current/images/netboot/mini.iso
> (SUCCESS)

Until an alpha release of the installer finds its way to unstable, I'd
advise using daily builds:
  https://d-i.debian.org/daily-images/daily-build-overview.html
  https://d-i.debian.org/daily-images/amd64/

Note that the netboot mini.iso builds fine, but using lvm2 at runtime is
likely to be problematic due to:
  https://bugs.debian.org/1015174


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#1015172: marked as done (wiki.debian.org: Sid installation fails - No kernel modules found - Wiki issue)

[Cyril Brulebois]

Debian Bug Tracking System  (2022-07-17):
> This is a temporary issue that always happens after Linux kernel
> updates and then gets automatically fixed after some time. In this case
> there was also a cron job delay so it took a bit longer. The Debian
> Installer team say this should be fixed in the next few hours.
> 
> PS: the debian-boot list would have been the place to report this.

Sorry for the incomplete analysis I provided Paul with, the kernel issue
mismatch is usually temporary, but that's not a factor here: there seem
to be some weird things going on, mixing and matching bits from bullseye
and unstable?

Anyway, there's another issue for d-i builds in unstable currently:
lvm2-udeb is not installable, which is why some builds are failing. I've
filed a bug report but it's not showing up just yet.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Tons of News html files under www.debian.org

[Cyril Brulebois]

Holger Wansing  (2022-02-03):
> while looking for the build status of the latest installation-guide
> upload for the website, I found hundreds of News html files residing
> directly under www.debian.org, starting from 1997 until 01.06.2018.
> 
> Below a loose excerpt of those (1 per year):
> 
> www.debian.org/19970301.en.html
> www.debian.org/19980101.en.html
> www.debian.org/19990212.en.html
> www.debian.org/2116.en.html
> www.debian.org/20010415.en.html
> www.debian.org/20020403.en.html
> www.debian.org/20030531.en.html
> www.debian.org/20040524.en.html
> www.debian.org/20050724.en.html
> www.debian.org/20060901.en.html
> www.debian.org/20070407.en.html
> www.debian.org/20080726.en.html
> www.debian.org/20090730.en.html
> www.debian.org/20100806.en.html
> www.debian.org/20110515.en.html
> www.debian.org/20120706.en.html
> www.debian.org/20130809.en.html
> www.debian.org/20141020.en.html
> www.debian.org/20150110.en.html
> www.debian.org/2016060402.en.html
> www.debian.org/20171007.en.html
> www.debian.org/20180310.en.html
> www.debian.org/20180601.en.html
> 
> They can be found on wolkenstein under /srv/www.debian.org/www and
> have all been created on 02. or 04. June 2018.
> 
> I guess there was some build script for the website completely going
> crazy and thus causing this.

End of May/beginning of June 2018 is when the cvs to git transition took
place, that could have been a side effect.

They were a bunch of commits on 2018-06-04 that seem like they could
have fixed the buggy behaviour you've spotted: you can check
6e1be1595a987b4c8b38df41146890869855f6a5 and a bunch of follow-up
commits if you like.

Anyway, the clean-up you're proposing looks good to me.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Site: packages.debian.org

[Cyril Brulebois]

Hallo Mechtilde,

Mechtilde Stehmann  (2021-08-18):
> after releasing Debian 11 (bullseye I want to follow the migration from
> sid to the new testing (Bookworm) at packages.debian.org

Usually tracker.debian.org is a nice(r) entry point to follow one
package's progress across suites.

> I hope you can refresh the sites so it is seen there.
> 
> If this isn't the right place to inform you, please give me a hint.

That's fine, and there's even a bug report already:
  https://bugs.debian.org/992258


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#992345: www.debian.org: bullseye release-notes are misleading about the needed disk space

[Cyril Brulebois]

Hi Vincent,

Vincent Lefevre  (2021-08-17):
> 649 upgraded, 113 newly installed, 11 to remove and 1 not upgraded.
> Need to get 455 MB of archives.
> After this operation, 558 MB of additional disk space will be used.
> 
> and I had around 650 MB free space

455+558 >> 650 so it was very unlikely to work?

But yeah, even if the release notes mention “After the download […]”
things, it could probably point out that you should look at both
numbers, instead of just using xx.x and AAA placeholders.

> I suspect that some packages like this one need a lot of temporary
> disk space (could this be related to initrd before compression?).
> But there is nothing about that in the release notes. And perhaps
> a warning should have been output by apt.

There's one for the directory where debs are downloaded (which you
didn't get since 455 << 650), but I suspect it's harder to estimate
where the whole “additional disk space” is going to end up, depending on
file system layout, etc.?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#989132: buster-pu: package wml/2.12.2~ds1-3~deb10u1

[Cyril Brulebois]

Package: release.debian.org
Severity: normal
Tags: buster
User: release.debian@packages.debian.org
Usertags: pu
X-Debbugs-Cc: Axel Beckert , debian-www@lists.debian.org

Hi,

(a...@debian.org in x-d-cc, who agreed with my helping on this topic, and
debian-www@lists.debian.org for information)


[ Reason ]

The wml package in buster contains a regression from stretch that leads
to various Unicode-related fun. It can trigger Unicode validity issues
in Chinese, which was seen and worked around for the build of the Debian
website; but it can also misrender various languages, if a non-ASCII
character happens to be the last one on a line in the WML source. That
includes the rather frequent word “à” in French (affecting hundreds of
pages on the Debian website), or “υ” as the last letter of the last word
(seen in Greek). This was also reported for Russian.

Patching the Debian website to avoid running into these situations could
be feasible but would also be impractical, as new/updated translations
would have to be monitored. And that wouldn't fix the rendering of
unsuspecting wml users outside the Debian website use case.

Patching wml instead was discussed in this MR against webmaster-team's
webwml, which includes some example of bad rendering, and many more data
points down the line (which are summed up below):
  https://salsa.debian.org/webmaster-team/webwml/-/merge_requests/596


[ Impact ]

Broken rendering when non-ASCII characters appear at the end of a line
in WML sources, which might be non-obvious (this wouldn't break a
build).


[ Tests ]

Obviously, I've used the Debian website as a “regression test” that
encompasses many files in various languages. My findings are available
there:
  
https://salsa.debian.org/webmaster-team/webwml/-/merge_requests/596#note_240902
  
https://salsa.debian.org/webmaster-team/webwml/-/merge_requests/596#note_240938

Basically, `file` can be used to determine whether rendering in
generated HTML files appears to be broken, mixing UTF-8 and
ISO-something (or similar) characters. With this, I confirmed that all
occurrences of “Non-ISO extended-ASCII” variations are being replaced
with full UTF-8 files (also variations, depending on long lines etc.).

I've also checked the expected changes are happening, with “broken
character” being replaced by “à” many many times in French (we have 700+
affected pages for that language alone). Non-HTML files don't appear to
change much either, as expected (those were inspected via diff, rather
than counting on file's output).

The corpus of generated HTML is 64466 files, which seems decent enough
as a real-life regression test…

Finally, I've checked that *only with the patched wml package*,
reverting the workaround that was put in place for Chinese doesn't break
the HTML generation again, and even gets us a better rendering than with
the workaround. More details in:
  
https://salsa.debian.org/webmaster-team/webwml/-/merge_requests/596#note_240938


[ Risks ]

I cannot say it will not regress or slightly change the output for some
specific users/files, but I would be quite surprised to see people show
up and complain that we fixed broken rendering…


[ Checklist ]

  [x] *all* changes are documented in the d/changelog
  [x] I reviewed all changes and I approve them
  [x] attach debdiff against the package in stable
  [x] the issue is verified as fixed in unstable


[ Changes ]

The package in buster is 2.12.2~ds1-2 (through an upload to unstable
that migrated into testing), the issue was fixed in the following upload
(2.12.2~ds1-3) which happened 1+ year later, with just a single patch.
I'm proposing to backport this specific upload to buster, hence the
rather obnoxious 2.12.2~ds1-3~deb10u1 version number. I've also
considered 2.12.2~ds1-2+deb10u1 which didn't look much better (and I'm
not sure going with 2.12.2~ds1-4 for cosmetic reasons would be
reasonable).


Thanks for considering!


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
diff -Nru wml-2.12.2~ds1/debian/changelog wml-2.12.2~ds1/debian/changelog
--- wml-2.12.2~ds1/debian/changelog 2019-02-17 18:39:38.0 +0100
+++ wml-2.12.2~ds1/debian/changelog 2021-05-25 05:47:04.0 +0200
@@ -1,3 +1,20 @@
+wml (2.12.2~ds1-3~deb10u1) buster; urgency=medium
+
+  * Backport Unicode fix to buster, fixing rendering issues with e.g.
+non-ASCII characters in various languages, as seen when building the
+Debian website. Some examples include ‘υ’ in Greek and ‘à’ in French
+when those characters are at the end of a line.
+
+ -- Cyril Brulebois   Tue, 25 May 2021 05:47:04 +0200
+
+wml (2.12.2~ds1-3) unstable; urgency=medium
+
+  * Add patch to fix regression in Unicode handling (especially Chinese)
+of "htmlstrip -O2" from Stretch to Buster by adding "no feature
+'unicode_strings'". (Closes: #959761)
+
+ -- Axel Beckert   Tue, 05 May 

Re: How can fix a typo in packages description?

[Cyril Brulebois]

Hello,

sebul  (2021-03-07):
> I found a typo at
> https://packages.debian.org/ko/buster/task-korean
> 설 치합니다.
> is incorrect
> 설치합니다.
> is correct
> How can I fix it ?

I think you may want to look at:
  https://ddtp.debian.org/
  https://www.debian.org/international/l10n/ddtp


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Chinese+Dutch: Fwd: webwml failed

[Cyril Brulebois]

Laura Arjona Reina  (2021-02-23):
> Something is wrong in one Chinese file

I suppose the translation-check should reference a commit that actually
touches the reference file, which doesn't seem to be the case for
7ada016b1391eef49f523db729cd5e481d676d62, which touched translations.
I'm cc-ing Wu XiangCheng who introduced those files, so that they can be
fixed. I see those two pointing at that commit:

chinese/devel/website/errors/404.wml
chinese/devel/website/thankyou.wml

The former might want to point at: 8e99b7f329aa779c90f21c78efc2b78b8a4583e3
The latter might want to point at: 1fed65f277a53d12839ebbf4454d9f7ea9fca5ed

but I'd rather see that checked by the translator. :)

> and one (different) Dutch file when building the website, below you
> can find the error log.

Should be fixed by 5a36f60cb5237da9fabb4c16b00c643ea5c9a24d.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: questions regading some encoding troubles in webwml

[Cyril Brulebois]

Bonjour Patrice,

Patrice Duroux  (2021-02-22):
> After observing the following broken char here:
> Introduction � l'empaquetage Debian
> <https://www.debian.org/doc/devel-manuals#packaging-tutorial> in
> https://www.debian.org/doc/index.fr.html
> 
> For that I went to its source file in the salsa module. But:
> 1. the rendering in salsa is surprising because it seems to handle a
> mixed-encoding situation, and so here we cannot see the mismatch:
> https://salsa.debian.org/webmaster-team/webwml/-/blob/master/french/doc/index.wml
> 2. I also observed some encoding variations regarding the apostrophe
> character that sometimes is the classic (U+0027) or the (U+2019). In my
> opinion, an apostrophe should be U+0027 and U+2019 uses for the right close
> single quotation mark.
> 
> So have Debian any rules/recommendations on the formal content?
> Any tool used to validate before (or after) any changes? (like a testcheck
> in its make?)
> Otherwise may I proceed directly using the edition facility of salsa?

This is a rendering issue with wml/buster:
  https://salsa.debian.org/webmaster-team/webwml/-/merge_requests/596

Hopefully that'll go away when a fixed version is deployed on the
machine building the website.

Cc-ing Axel as a gentle reminder. ;)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#924172: www.debian.org: differences under english/ between builds in stretch and buster

[Cyril Brulebois]

Cyril Brulebois  (2020-06-08):
> Dropping the ../english/ prefix for sitemap.wml leads to a reasonable
> diff for the English sitemap (sitemap.en.html):
> 
> --- stretch.en.html   2020-06-08 00:07:43.916768223 +0200
> +++ buster.en.html2020-06-08 00:07:47.248781809 +0200
> @@ -4,8 +4,8 @@
>
>Site map for Debian web pages 
>mailto:webmas...@debian.org;>
> -  
> -  
> +  
> +  
>
>
>
> @@ -377,7 +377,7 @@
>  
>  Last Modified: Sun, Jun 7 21:49:47 UTC 2020
>  
> -Last Built: Sun, Jun 7 22:07:43 UTC 2020
> +Last Built: Sun, Jun 7 22:07:46 UTC 2020
>
>Copyright  1997-2020
>   https://www.spi-inc.org/;>SPI and others; See  href="./license" rel="copyright">license terms
> 
> Therefore, I suspect we *might* be able to get away with this if we were
> to add a symlink from all language directories, from $LANG/sitemap.wml
> to ../english/sitemap.wml, as a companion change to dropping the
> ../english prefix mentioned above (only tested successfully on English).
> I'll be checking this hypothesis and possibly pushing a branch / opening
> an MR if that works fine.

OK. This isn't so simple. I've pushed this branch:
  
https://salsa.debian.org/webmaster-team/webwml/-/tree/pu/partial-workaround-924172

I'm attaching several sitemap files, built either in stretch or in
buster, in English or in French, from the master branch or from the
workaround branch:

buster.en.html.master
buster.en.html.workaround
buster.fr.html.master
buster.fr.html.workaround
stretch.en.html.master
stretch.en.html.workaround
stretch.fr.html.master
stretch.fr.html.workaround

For each language, diffing the stretch built between master and the
workaround yields no (practical) differences.

Diffing stretch.en.master.html and buster.en.master.html shows many
../english/ being added; which could be arguably seen as a no-op.

Diffing stretch.fr.master.html and buster.fr.master.html shows many
../english/ being added; which breaks navigation as that's another
language…

Diffing stretch.en.master.html and buster.en.workaround.html shows
no practical differences (per my previous mail).

Diffing stretch.fr.master.html and buster.fr.workaround.html shows
the lost translations between (patched or unpatched) French version
built in stretch and patched French version built in English. It
seems we're losing translations for items which would have suffered
from the ../english/ addition.

Interestingly, this list seems to match the special cases defined in
english/sitemap.wml (which is about pass protection this time, see
the escape tag definition and comment above). I haven't tried to
double check the flags passed to each pass in each cases though
(running out of time).


Overall, it seems my proposed patch papers around the issue but not
fully. But it could help unstuck the move to buster if we agree the
partial translations for the sitemaps are better than rather broken
sitemaps.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#924172: www.debian.org: differences under english/ between builds in stretch and buster

[Cyril Brulebois]

in the debian-flyers repo?)
> 
> 4. Changes in ordering of coordinators
> 5. Changes in ordering under wnpp
> 6. Changes in order under l10n
> 8. More ordering changes (architectures, DSAs)
> 
> Thanks for reporting, and for the work towards reproducibility.
> I think these are not blockers for the migration to buster of
> www-master.debian.org
> 
> Maybe we could open a specific bug about these "reproducibility issues"
> and see if somebody is willing/able to work on it?

I'm very fine with such a thing. I just wanted to classify differences
between stuff that were possibly unimportant and stuff that might need
fixing.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

www.debian.org: please reduce entropy / make build more deterministic

[Cyril Brulebois]

[ forwarding without the first patch which made this mail too big for
  the mailing list, and for which Laura has a real solution anyway;
  filed in the BTS as #926550 ]

Cyril Brulebois (2019-04-06):
| Package: www.debian.org
| Severity: normal
| User: www.debian@packages.debian.org
| Usertags: scripts
| 
| Hi,
| 
| As seen in other bug reports, I've been looking at builds within stretch
| and buster. I've started a branch called “fight-entropy” in which I've
| reworked Makefiles so that one can get a more deterministic order when
| building the website from scratch, with -j1 (to disable parallelism).
| 
| I've also applied workarounds for #924333 and #924365 (both attached),
| so that the build finishes successfully.
| 
| You'll find attached the diff of filtered build logs. Those are post
| processed to get rid of the path to the checkout (stretch vs. buster),
| and to suppress wget output.
| 
| There are a couple of different touch calls, some differences of
| versions for tools that are used to build documents (hello, LaTeX2e,
| Babel, dvips!), and english/international/l10n/scripts/gen-files.pl
| would likely benefit from an extra sort or two. But at least the build
| logs with -j1 are much more easily reviewable with the fight-entropy
| branch.
| 
| Slightly more worrisome: some encoding related issues (“Invalid UTF8” or
| “Malformed UTF-8 character”) that might need to be addressed.
| 
| I still have to investigate the differences between generated files
| (stretch vs. buster), but results will be reported in a separate bug
| report.
| 
| All in all, please consider merging:
|   https://salsa.debian.org/webmaster-team/webwml/commits/fight-entropy
| 
| and maybe consider the attached workaround patches as well.

I might have mentioned that on IRC only: build time gets a big bump:
270 minutes in stretch, 350 minutes in buster (keeping in mind that's
with a single core anyway; but the performance penalty is real…).


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant
>From 47a1c96f743d970aa1a52b35b3d0f7ebd9c88bc8 Mon Sep 17 00:00:00 2001
From: Cyril Brulebois 
Date: Sat, 6 Apr 2019 12:21:54 +
Subject: [PATCH 2/2] Disable turkish which doesn't build on buster (#924365).

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index d368e08b785..800795450eb 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ LANGUAGES := english albanian arabic armenian bulgarian catalan 
chinese croatian
  danish dutch esperanto finnish french galician german greek 
hebrew \
  hungarian indonesian italian japanese korean lithuanian \
  norwegian persian polish portuguese romanian russian slovak 
slovene \
- spanish swedish tamil turkish ukrainian vietnamese
+ spanish swedish tamil ukrainian vietnamese
 
 LANGUAGES-install := $(addsuffix -install,$(LANGUAGES))
 LANGUAGES-clean := $(addsuffix -clean,$(LANGUAGES))
-- 
2.11.0

--- stretch/stretch-j1.log.filt.wget	2019-04-06 20:19:48.198683352 +0200
+++ buster/buster-j1.log.filt.wget	2019-04-06 20:19:55.210799691 +0200
@@ -4,6 +4,8 @@
 make[2]: Entering directory '??ROOT??webwml/english/po'
 make[2]: Nothing to be done for 'install'.
 make[2]: Leaving directory '??ROOT??webwml/english/po'
+touch ../english/template/debian/basic.wml # because of navbar.wml language_names.wml footer.wml
+touch ../english/template/debian/template.wml # because of basic.wml
 (cd ./.. && ./build_vcs_cache.pl)
 Initialising VCS cache for performance
  ... done
@@ -13,6 +15,7 @@
 ./../touch_translations.pl ??ROOT??webwml/english/contact.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:donations.en.html@g+w   donations.wml
 ./../touch_translations.pl ??ROOT??webwml/english/donations.wml en
+touch ../english/template/debian/mainpage.wml # because of basic.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:index.en.html@g+w   index.wml
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:license.en.html@g+w   license.wml
 ./../touch_translations.pl ??ROOT??webwml/english/license.wml en
@@ -32,6 +35,7 @@
 make[2]: Entering directory '??ROOT??webwml/english/Bugs'
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:Access.en.html@g+w   Access.wml
 ../../touch_translations.pl ??ROOT??webwml/english/Bugs/Access.wml en
+touch ../../english/Bugs/pkgreport-opts.inc # because of common_tags.wml templates.pot countries.pot langs.pot date.pot bugs.pot
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:Developer.en.html@g+w   Developer.wml
 ../../touch_translations.pl ??ROOT??webwml/english/Bugs/Developer.wml en
 wml -q -D CUR_YEAR=2019 -o UNDEFuEN:Reporting.en.html@g+w   Reporting.wml
@@ -52,6 +56,7 @@
 make[2]: Leaving directory '??ROOT??webwml/english/Bugs'
 make -C CD
 make[2]: Entering directory '??ROOT??webwml/english/CD'
+touch ../../english/template/d

Re: The website is not building correctly (git pull stalled)

[Cyril Brulebois]

Hi,

Laura Arjona Reina  (2019-04-04):
> El 4/4/19 a las 19:54, Boyuan Yang escribió:
> > Is there any possibility that we do a "git reset --hard HEAD; git
> > clean -xdf" before actually "git pull"?
> > 
> 
> My git skills are not enough to know if this could have unintended
> consequences in www-master.debian.org, or if it would penalise the
> build time. I tend to think that if a normal "git pull" fails, we'd
> like to have a look at where is the issue, as in this case.
> 
> In any case, I have created a merge request with your suggestions as
> addition to the 2git script in our cron repo:
> 
> https://salsa.debian.org/webmaster-team/cron/merge_requests/3
> 
> I hope others can merge or comment on that.

The first command is similar to “git checkout -f” → forgetting about
all local changes; the second command is basically about removing all
untracked files, meaning killing all generated files.

This would make sure there are no leftovers, ever. Or let us detect
issues (like some that I reported a few weeks ago) that could be
hidden if a build had happened previously. But the build time
penalties don't seem worth it?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: old https security debian org does not redirect to new page, invalid ssl cert instead

[Cyril Brulebois]

Hi,

dra...@peerfreedom.org  (2019-03-30):
> Previously the web page for debian security was located on:
> 
> security.debian.org
> 
> Going there now through http, will redirect to the new address
> www.debian.org/security/
> 
> But going there through https, will instead return SSL errror, cert by
> unknown CA "Debian SMTP CA" which seems to have "CA key identifier" "00
> f6 08 13 4a 49 f7 da d3",
> entire cert sha256 fingerprint is
> "DF:68:20:DA:43:5A:7C:1A:9C:43:8B:56:56:24:92:A5:E6:EC:F6:B1:92:8F:D1:4C:9F:5D:93:C7:7D:13:26:1B"

Not all things served over http:// are supposed to be reachable over
https:// so that might be just considered as not-a-bug.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#924365: www.debian.org: FTBFS in buster: turkish vs. tr_TR locale

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

The turkish subdirectory doesn't build in buster, apparently due to
possible locale issues:

(buster-amd64-webwml)kibi@wodi:~/debian-www/buster/webwml$ make -j1 -C 
turkish STRICT_ERROR_CHECKS=1 USE_SAMPLE_FILES=1
make: Entering directory '/home/kibi/debian-www/buster/webwml/turkish'
make -C po install
make[1]: Entering directory '/home/kibi/debian-www/buster/webwml/turkish/po'
make[1]: Leaving directory '/home/kibi/debian-www/buster/webwml/turkish/po'
wml -q -D CUR_YEAR=2019 -o UNDEFuTR:index.tr.html@g+w   index.wml
ePerl:Error: Perl runtime error (interpreter rc=0)

 Contents of STDERR channel: -
Locale 'tr_TR' may not work well.
The following characters (and maybe others) may not have the same meaning 
as the Perl program expects:
I i
; codeset=ISO-8859-9
--
** WML:Break: Error in Pass 3 (rc=1).
Died at /usr/share/wml/TheWML/Frontends/Wml/Runner.pm line 403.

TheWML::Frontends::Wml::Runner::_run_pass(TheWML::Frontends::Wml::Runner=HASH(0x55db148af458),
 3, SCALAR(0x55db14e351b8), SCALAR(0x55db14e35188), SCALAR(0x55db14e351a0)) 
called at /usr/share/wml/TheWML/Frontends/Wml/Runner.pm line 441

TheWML::Frontends::Wml::Runner::_passes_loop(TheWML::Frontends::Wml::Runner=HASH(0x55db148af458))
 called at /usr/share/wml/TheWML/Frontends/Wml/Runner.pm line 726

TheWML::Frontends::Wml::Runner::_output_and_cleanup(TheWML::Frontends::Wml::Runner=HASH(0x55db148af458))
 called at /usr/share/wml/TheWML/Frontends/Wml/Runner.pm line 930

TheWML::Frontends::Wml::Runner::run_with_ARGV(TheWML::Frontends::Wml::Runner=HASH(0x55db148af458),
 HASH(0x55db14908ff8)) called at /usr/bin/wml line 47
make: *** [/home/kibi/debian-www/buster/webwml/english/Makefile:43: 
index.tr.html] Error 2
make: Leaving directory '/home/kibi/debian-www/buster/webwml/turkish'


Many languages are using UTF-8 locales, exceptions are the following ones:

$ git grep CUR_LOCALE -- */.wmlrc|grep -v UTF-8$|grep -v utf8$
armenian/.wmlrc:-D CUR_LOCALE=hy_AM
bulgarian/.wmlrc:-D CUR_LOCALE=bg_BG
dutch/.wmlrc:-D CUR_LOCALE=nl_NL
hungarian/.wmlrc:-D CUR_LOCALE=hu_HU
lithuanian/.wmlrc:-D CUR_LOCALE=lt_LT
persian/.wmlrc:-D CUR_LOCALE=fa_IR
romanian/.wmlrc:-D CUR_LOCALE=ro_RO
slovak/.wmlrc:-D CUR_LOCALE=sk_SK
slovene/.wmlrc:-D CUR_LOCALE=sl_SI
turkish/.wmlrc:-D CUR_LOCALE=tr_TR

I suppose contacting translators and checking with them whether ISO-8859-9
is the right encoding to use in our source files, and whether there might
be some issues with Perl, would be a starting point?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924333: FTBFS: missing czech/News/weekly/index.wml

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

The build fails both in stretch and in buster due to a missing file
(czech/News/weekly/index.wml):

make -C weekly
make[3]: Entering directory 
'/home/kibi/debian-www/stretch/webwml/czech/News/weekly'
make[3]: *** No rule to make target 'index.wml', needed by 'index.cs.html'. 
 Stop.
make[3]: Leaving directory 
'/home/kibi/debian-www/stretch/webwml/czech/News/weekly'
../../Makefile.common:79: recipe for target 'weekly' failed
make[2]: *** [weekly] Error 2
make[2]: Leaving directory '/home/kibi/debian-www/stretch/webwml/czech/News'
../Makefile.common:79: recipe for target 'News' failed
make[1]: *** [News] Error 2
make[1]: Leaving directory '/home/kibi/debian-www/stretch/webwml/czech'
Makefile:28: recipe for target 'czech' failed
make: *** [czech] Error 2

This seems to date back to this commit:
  
https://salsa.debian.org/webmaster-team/webwml/commit/0cbd1d8ebfa0d33b09d84e495a0c661e8830dabd

which is from 2007.

If fixing the missing index file doesn't make sense, the parent Makefile
should perhaps not recurse into that subdirectory?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924139: www.debian.org: migrate from python to python3

[Cyril Brulebois]

Hi,

Please keep the submitter in copy when replying on the BTS.

황병희  (2019-03-10):
> Well i'm not position to resolve it. However that is very very very
> dangerous i think. Anyhow, it is interesting PR, to me.

Why would porting to a new version be “dangerous”?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Intent to upload new WML version to unstable for buster

[Cyril Brulebois]

Hi Axel,

Axel Beckert  (2018-12-25):
> wml package maintainer here.
> 
> For multiple Debian releases I've kept newer WML releases in
> experimental as they crashed rather often with my own WML based
> projects.
> 
> I recently uploaded 2.10.1 to experimental which seems to work
> identical to 2.0.12. I just had to include a few patches for making
> wmk and some corner cases work properly again. And with them works
> well on my own projects as well as with webwml (tried English, German
> and Japanese).
> 
> All were built without glitches and english took nearly an hour to
> complete on my laptop, but did complete without a single non-zero exit
> code.
> 
> With the exception of a bunch of CSS paths deeper down, the website
> generated with it looks fine to me.
> 
> So I intent to upload the just released version 2.12.0 (which
> incorporates all the fixes I had to put into the Debian package) to
> unstable before New Year's eve.
> 
> I'd be happy if someone else could also do some checks to ensure the
> WWW team won't have too many headaches when www.debian.org (or
> whereever the pages are actually built) gets upgraded to buster.

Thanks for the heads-up.

I've filed a bunch of bug reports against www.debian.org according to my
findings, a lot of them being about leftovers or various things related
to the build system. Changes likely linked to the wml update are in the
last bug report:

#924135: clean-up: old .cvsignore files
#924136: README: mention at least make
#924137: www.debian.org: debian.org metapackage: add python python-lxml
#924139: www.debian.org: migrate from python to python3
#924140: README: please mention STRICT_ERROR_CHECKS=1 USE_SAMPLE_FILES=1
#924143: www.debian.org: debian.org metapackage: add dependency for 
Locale::Codes
#924144: www.debian.org: debian.org metapackage: missing libgd-perl 
dependency
#924146: www.debian.org: possible leftover: 
norwegian/Bugs/pseudo-packages.inc
#924172: www.debian.org: differences under english/ between builds in 
stretch and buster

Now looking at making it easier to compare a full build instead of just
a single directory.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#924172: www.debian.org: differences under english/ between builds in stretch and buster

[Cyril Brulebois]

ebwml/english/security/ref-table.inc



Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924146: www.debian.org: possible leftover: norwegian/Bugs/pseudo-packages.inc

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

Tracking down another issue, I've spotted that one particular language has an 
inc file that appears to be an older copy of a inc file that's otherwise only 
in the english/ directory:

norwegian/Bugs/pseudo-packages.inc

As the following file correctly references (includes) the file under
english/ instead of the one under norwegian/ (which is what is done for
other languages as well), it might be safe to simply delete it? I didn't
double check that though.

norwegian/Bugs/pseudo-packages.wml


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924143: www.debian.org: debian.org metapackage: add dependency for Locale::Codes

[Cyril Brulebois]

Cyril Brulebois  (2019-03-09):
> When building on buster, the Perl upgrade changes at least one thing:
> one needs to install an extra package to get the Locale::Codes module.

For completeness, the error message:

ePerl:Error: Perl runtime error (interpreter rc=0)

 Contents of STDERR channel: -
Locale::Codes will be removed from the Perl core distribution in the next 
major release. Please install the separate liblocale-codes-perl package. It is 
being used at /usr/share/perl/5.28/Locale/Language.pm, line 22.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#924144: www.debian.org: debian.org metapackage: missing libgd-perl dependency

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

Setting up a minimal chroot with dependencies from the metapackage, the
GD perl module cannot be found. It might not have been an issue yet as
some other metapackages can pull libgd*-perl packages, but it seems to
make sense to document the dependency explicitly anyway.

I don't seem to have the error message or the using script handy, but I
can double check and provide that extra piece of information if that's
desired.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924143: www.debian.org: debian.org metapackage: add dependency for Locale::Codes

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

When building on buster, the Perl upgrade changes at least one thing:
one needs to install an extra package to get the Locale::Codes module.
It used to be shipped under perl-modules, but now has its own separate
package.

In stretch, it's provided by:
  perl-modules
  perl-modules-5.24

In testing/sid, it's shipped in:
  liblocale-codes-perl | 3.59-1| testing| source, all
  liblocale-codes-perl | 3.60-1| unstable   | source, all

so it looks to me the debian.org metapackage could gain a dependency
like the following, to ensure the extra package gets installed when
moving to buster?

  liblocale-codes-perl | perl-modules-5.24

which should be installable in stretch and allowing a transition to
the real package (liblocale-codes-perl) when the host gets dist
upgraded to buster?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924140: README: please mention STRICT_ERROR_CHECKS=1 USE_SAMPLE_FILES=1

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

It was a bit surprising to see that many errors, either due to missing
dependencies, or missing data, were not leading to a global failure when
the top-level make returns.

It was also a bit surprising to see that some targets couldn't get
processed successfully.

It turned out that the former issue is at least partiallly fixed by
setting STRICT_ERROR_CHECKS=1; the latter can be worked around by
setting USE_SAMPLE_FILES=1, which sets up some sample data files for
files that would otherwise be set up by some sync scripts getting run
from cron. Examples include:

$(ENGLISHSRCDIR)/devel/people.names
$(DATADIR)/Maintainers
$(ENGLISHSRCDIR)/$(CUR_DIR)/data/%
$(L10N_DIR)/data/%
$(ENGLISHDIR)/mirror/Mirrors.masterlist

That was implemented in:
  
https://salsa.debian.org/webmaster-team/webwml/commit/f0290accd46ee45383b56b9acc9624f5707f6b0e

(Thanks, Wouter!)

It seems to me it would make sense to mention both variables in the
top-level README, so that people interested in working on a full build
wouldn't have to figure out why some targets seem to be missing or not
defined, and to track down individual READMEs like:

 english/mirror/Mirrors.masterlist.README

FWIW, that's also what's getting used for CI, see:

README_CI.gitlab-ci.yml


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924139: www.debian.org: migrate from python to python3

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

There are a couple of scripts that rely on Python at the moment:

$ find -name '*.py'|sort
./cvsup.py
./english/mirror/timestamps/archive_mirror_check.py
./english/mirror/timestamps/mirror_check.py
./english/security/oval/generate.py
./english/security/oval/oval/__init__.py
./english/security/oval/oval/definition/__init__.py
./english/security/oval/oval/definition/differ.py
./english/security/oval/oval/definition/generator.py
./english/security/oval/oval/parser/__init__.py
./english/security/oval/oval/parser/dsa.py
./english/security/oval/oval/parser/wml.py

It might be possible to get rid of the first one after the CVS→Git
migration, but others might need to get ported to Python 3 at some
point, as Python 2 is on the way out.

The debian.org metapackage for the website will need to get updated
accordingly during/after the migration.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924136: README: mention at least make

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

The README only mentions a couple of packages, along with a link to the
debian.org meta package for the website.

It would make sense to have at least make mentioned in README as well,
as that's the entry point for building the website?


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924137: www.debian.org: debian.org metapackage: add python python-lxml

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

The metapackage (in debian.org.git) doesn't mention either python or
python-lxml, but both are needed for the
english/security/oval/oval/definition/generator.py script.

This should be fixed by submitting a fix to DSA, but I'm failing this
against www.debian.org for the time being as part of my “sprint”.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#924135: clean-up: old .cvsignore files

[Cyril Brulebois]

Package: www.debian.org
Severity: normal
User: www.debian@packages.debian.org
Usertags: scripts

Hi,

While working on rebuilding the website within stretch and buster, I've
noticed the following.

There are many leftovers from CVS times in the tree, namely .cvsignore
files:

$ find -name .cvsignore|sort
./arabic/po/.cvsignore
./armenian/po/.cvsignore
./bulgarian/po/.cvsignore
./catalan/MailingLists/.cvsignore
./catalan/po/.cvsignore
./chinese/MailingLists/.cvsignore
./chinese/po/.cvsignore
./croatian/MailingLists/.cvsignore
./croatian/po/.cvsignore
./czech/po/.cvsignore
./danish/MailingLists/.cvsignore
./danish/po/.cvsignore
./dutch/po/.cvsignore
./english/Bugs/.cvsignore
./english/MailingLists/.cvsignore
./english/devel/.cvsignore
./english/devel/misc/.cvsignore
./english/devel/wnpp/.cvsignore
./english/international/l10n/data/.cvsignore
./english/mirror/.cvsignore
./english/po/.cvsignore
./esperanto/po/.cvsignore
./finnish/MailingLists/.cvsignore
./finnish/po/.cvsignore
./french/MailingLists/.cvsignore
./french/po/.cvsignore
./german/MailingLists/.cvsignore
./german/po/.cvsignore
./greek/po/.cvsignore
./hebrew/po/.cvsignore
./hungarian/MailingLists/.cvsignore
./hungarian/po/.cvsignore
./indonesian/po/.cvsignore
./italian/po/.cvsignore
./japanese/po/.cvsignore
./korean/po/.cvsignore
./lithuanian/po/.cvsignore
./norwegian/MailingLists/.cvsignore
./norwegian/po/.cvsignore
./persian/po/.cvsignore
./polish/po/.cvsignore
./portuguese/po/.cvsignore
./romanian/po/.cvsignore
./russian/MailingLists/.cvsignore
./russian/po/.cvsignore
./slovak/po/.cvsignore
./slovene/po/.cvsignore
./spanish/po/.cvsignore
./swedish/po/.cvsignore
./tamil/po/.cvsignore
./turkish/po/.cvsignore
./ukrainian/po/.cvsignore
./vietnamese/po/.cvsignore

It might make sense to merge some of it into .gitignore, but getting rid
of them in the end would make sense I suppose.


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

Bug#884229: Bug #884229: [packages.debian.org] 500 error/Internal Server Error (in packages-pkgmirror-csail.debian.org)

[Cyril Brulebois]

Hi Olly,

Olly Betts <o...@survex.com> (2018-01-26):
> Yes.  The directory on pkgmirror-csail actually had two different
> versions of the database using different database backends, so the
> files which aren't in the directory on picconi are essentially an
> old version of the same database.
> 
> That alone shouldn't cause the search to fail though - it should just
> open one or the other (it looks like both Xapian 1.2.x and 1.4.x will
> open the old one in this rather odd situation).
> 
> Xapian's replication should cleanly handle the database being replicated
> switching backends so I'm guessing the replication here is using rsync
> (without --delete-after) or similar?  That might also explain why the
> old database is broken as the databases aren't safe to rsync if an
> update is in progress - if the last rsync of the old database happened
> while an update was in progress, it could have been left broken, which
> would typically result in particular searches failing.
> 
> Updating with rsync can also break searches on the mirror while the
> rsync is in progress even if the master isn't updating, so perhaps we
> should switch to using Xapian's replication?  That's already being
> used for the main website search (I set it up with weasel at Debconf
> 15).
> 
> Happy to assist with that, though I'll probably need ssh access to
> pkgmirror-csail (seems I currently only have it for picconi).  My
> Debian username is "olly".

Looking around without knowing much about pkg stuff:
 - picconi has: conf/pushpdo.mirror → csail pkgmirror-csail.debian.org pkg_user
 - those bits are read by bin/pushpdo, which calls rsync this way:
 rsync -e "ssh -i ${MKEYFILE} -${MPROTO} ${SSH_OPTS}" -av --stats  
"${MIRRORPATH}"  ${MUSER}@${MHOSTNAME}:/does/not/matter 
>"${LOGDIR}/pushpdo.${MLNAME}.log"

I suppose help is welcome to possibly fix/improve that, but I'm no pkg
expert (I just happened to have helped one or twice in the past, and to
still be in the pkg_maint group…)


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Bug#884229: Bug #884229: [packages.debian.org] 500 error/Internal Server Error (in packages-pkgmirror-csail.debian.org)

[Cyril Brulebois]

Control: tag -1 patch

Hi Laura,

Thanks for mentioning this on #-www today!

Laura Arjona Reina <larj...@debian.org> (2018-01-24):
> I can reproduce the problem with some other packages (I tried several from the
> list https://packages.debian.org/unstable/main/newpkg and got the error in the
> links to buster and sid).
> 
> I also tried
> https://packages.debian.org/stretch-backports/kernel-image-4.14.0-0.bpo.2-arm64-di
> with the same results (error 500).
> 
> Thanks to some help in the IRC channel we've found that only one of our wo
> machines serving packages.debian.org fails:
> 
> https://packages-pkgmirror-csail.debian.org/sid/elpa-find-file-in-project
> gives the 500 Internal Server Error
> but
> https://packages-picconi.debian.org/sid/elpa-find-file-in-project
> works.
> 
> If I can be of any help to solve this, just tell. I'll try to look at the logs
> later, but I'm not sure I have the corresponding permissions (will try).

From the apache error log on the mirror:
> mod_fcgid: stderr: Can't call method "get_document" on an undefined value at 
> ../lib/Packages/Search.pm line 264.
> End of script output before headers: dispatcher.fcgi

Looking at the code, that's a method call on a xapian query result.

Looking at the xapian directory on picconi:
> pkg_user@picconi:/srv/packages.debian.org$ ls -l files/db/xapian/
> total 302384
> -rw-r--r-- 1 pkg_user pkg_maint   1974272 Jan 24 15:56 docdata.glass
> -rw-r--r-- 1 pkg_user pkg_maint 0 Jan 24 15:53 flintlock
> -rw-r--r-- 1 pkg_user pkg_maint   138 Jan 24 15:56 iamglass
> -rw-r--r-- 1 pkg_user pkg_maint 181952512 Jan 24 15:56 position.glass
> -rw-r--r-- 1 pkg_user pkg_maint  80699392 Jan 24 15:56 postlist.glass
> -rw-r--r-- 1 pkg_user pkg_maint  44998656 Jan 24 15:56 termlist.glass

Looking at the xapian directory on the mirror:
> pkg_user@pkgmirror-csail:/srv/packages.debian.org$ ls -l files/db/xapian/
> total 537584
> -rw-r--r-- 1 pkg_user pkg_maint   1974272 Jan 24 09:48 docdata.glass
> -rw-r--r-- 1 pkg_user pkg_maint 0 Jan 24 09:45 flintlock
> -rw-r--r-- 1 pkg_user pkg_maint28 Nov  8 09:26 iamchert
> -rw-r--r-- 1 pkg_user pkg_maint   138 Jan 24 09:48 iamglass
> -rw-r--r-- 1 pkg_user pkg_maint  1699 Nov  8 09:30 position.baseA
> -rw-r--r-- 1 pkg_user pkg_maint  1725 Nov  8 09:30 position.baseB
> -rw-r--r-- 1 pkg_user pkg_maint 111788032 Nov  8 09:30 position.DB
> -rw-r--r-- 1 pkg_user pkg_maint 181944320 Jan 24 09:48 position.glass
> -rw-r--r-- 1 pkg_user pkg_maint  1254 Nov  8 09:30 postlist.baseA
> -rw-r--r-- 1 pkg_user pkg_maint  1263 Nov  8 09:30 postlist.baseB
> -rw-r--r-- 1 pkg_user pkg_maint  81616896 Nov  8 09:30 postlist.DB
> -rw-r--r-- 1 pkg_user pkg_maint  80707584 Jan 24 09:48 postlist.glass
> -rw-r--r-- 1 pkg_user pkg_maint54 Nov  8 09:30 record.baseA
> -rw-r--r-- 1 pkg_user pkg_maint56 Nov  8 09:30 record.baseB
> -rw-r--r-- 1 pkg_user pkg_maint   2523136 Nov  8 09:30 record.DB
> -rw-r--r-- 1 pkg_user pkg_maint   691 Nov  8 09:30 termlist.baseA
> -rw-r--r-- 1 pkg_user pkg_maint   703 Nov  8 09:30 termlist.baseB
> -rw-r--r-- 1 pkg_user pkg_maint  44883968 Nov  8 09:30 termlist.DB
> -rw-r--r-- 1 pkg_user pkg_maint  45006848 Jan 24 09:48 termlist.glass

For files dated Jan 24, the timestamps don't match, but that's probably
just a sync waiting to happen, and that doesn't explain the issues which
have been happening for so long. I suspected the stray files instead,
dating back to November. I've created an “old-files” directory and moved
them there, and the mirror seems to be behaving properly now.

I'm tagging this bug report with “pending” for the time being, to give
people a chance to comment. But I suppose it should be safe to remove
those old files entirely?

FTR, that would mean cleaning this:

230M /srv/packages.debian.org/files/db/xapian/old-files/


Cheers,
-- 
Cyril Brulebois (k...@debian.org)<https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant


signature.asc
Description: PGP signature

Re: Installation guide is not updated in some languages

[Cyril Brulebois]

Samuel Thibault  (2017-06-20):
> Samuel Thibault, on lun. 19 juin 2017 23:16:09 +0200, wrote:
> The buildweb.sh script is basically just forcing 
> 
> export official_build="1" (which is set for package upload anyway)
> export web_build="1"
> export manual_target="for_wdo"
> 
> 
> The only differences are
> 
> - these links on the website:
> 
> example preconfiguration file from  href="http://www.debian.org/releases/stretch/example-preseed.txt; 
> target="_top">http://www.debian.org/releases/stretch/example-preseed.txt.
> 
> which are made
> 
> example preconfiguration file from  href="../example-preseed.txt" target="_top">../example-preseed.txt.
> 
> in the package. I'd say we can use relative links on the website as
> well.

Looks good to me, yes.

> - the link to the various formats:
> 
> The document you are now reading, which is the official version of the
> Installation Guide for the stretch release of Debian; available
> in  href="http://www.debian.org/releases/stretch//installmanual; 
> target="_top">various formats and
> translations.

(nitpick: Probably a slash too many before installmanual.)

> in the website, and 
> 
> This document you are now reading, in plain ASCII, HTML or PDF format.
> 
> 
> 
> install.en.txt
> 
> 
> install.en.html
> 
> 
> install.en.pdf
> 
> 
> 
> in the package. The real difference is "and [various] translations".  I
> don't think this is worth making a difference between the website and
> the package?

I don't think it's required, but it might be nice to give doc/www
people a chance to voice their opinion here.


KiBi.


signature.asc
Description: Digital signature

Re: Installation guide is not updated in some languages

[Cyril Brulebois]

Samuel Thibault  (2017-06-16):
> Paul Wise, on mer. 14 juin 2017 11:02:11 +0800, wrote:
> > > (and it's not only a question of file renaming, links in the .html
> > > files need to be updated accordingly).
> > 
> > That would be the main issue.
> 
> Or not: we can simply change the rules that produce .html.en into
> producing .en.html, and drop the rules which produce .html

I didn't check yet, but do we have different contents in .html and
.CC.html? If we do, oops. If we don't, we could keep symlinks instead,
and ignore them when building the website?


KiBi.


signature.asc
Description: Digital signature

Bug#864970: www.debian.org: downloads page is missing download links

[Cyril Brulebois]

Control: tag -1 pending

Hi,

Stuart Prescott  (2017-06-18):
> Package: www.debian.org
> Severity: important
> 
> Hi!
> 
> in amongst all the excitement of the stretch release, the downloads page [1]
> has lost most of the download links. The only link currently displayed is
> the "multi-arch" image, which isn't being produced for 9.0 [2]
> 
> [1]  https://www.debian.org/CD/http-ftp/#stable

The missing define-tag for stretch-as-the-new-stable was just added to
installer.wml, and Paul rebuilt the website so that it's taken into
account. Marking this as pending…

(And note taken for the next release cycle.)

> [2]  https://wiki.debian.org/Teams/DebianCD/ReleaseTesting/Stretch (Errata #2)

Are we expected to see improvements in this area, or should we get rid
of the multi-arch dvd link as well? (… hence not closing this bug report
entirely). Added debian-cd@ to the loop.


KiBi.


signature.asc
Description: Digital signature

Re: urgent: help us fix the background for website banner for Stretch

[Cyril Brulebois]

Hi,

And thanks for your work on this. Please pretend I don't know anything
about web, standards, and best practices.

Laura Arjona Reina  (2017-06-17):
> But when I try to use those images (I made a test with the Inspector
> of Firefox), and I make ctrl-- to make size smaller so I emulate a big
> screen, I see that the background is repeated (this is ok) creating
> some "jumps" in the color gradient (this is not ok).
> 
> See https://lut.im/ncJMP6Sm9L/u7UvWdU57CSuQu3K.jpg for an example of
> what happens.
> 
> I don't know very well why this happens but I *think* it is because
> the "ending" colour at left edge and right edge is maybe not exactly
> the same.
> 
> I don't know how to fix that. If any of you can provide an improved
> image for background, it's very appreciated. If we don't get it soon,
> we'll use the one I committed and we'll try to improve it later.
> 
> If somebody starts to work on this, please reply to all so others
> don't repeat work.

We could just make the gradient image take the whole horizontal space,
that seems to work just fine on this 1920px wide screen.

Just set background-size to 100% and disabled the repeat mode, see
attached patch.


KiBi.
Index: debhome.css
===
RCS file: /cvs/webwml/webwml/english/debhome.css,v
retrieving revision 1.22
diff -u -r1.22 debhome.css
--- debhome.css	17 Jun 2017 17:52:04 -	1.22
+++ debhome.css	17 Jun 2017 18:29:39 -
@@ -26,7 +26,8 @@
 #splash {
 	background-image: url('Pics/softwaves_web_bg.png');
 	background-position:top center;
-	background-repeat: repeat;
+	background-repeat: no-repeat;
+	background-size: 100%;
 	margin-top: 0;
 	margin-bottom: 1.5em;
 	text-align: center;


signature.asc
Description: Digital signature

Re: Installation guide is not updated in some languages

[Cyril Brulebois]

Hi,

Samuel Thibault  (2017-06-05):
> Samuel Thibault, on dim. 04 juin 2017 12:08:18 +0200, wrote:
> > Samuel Thibault, on dim. 04 juin 2017 11:54:04 +0200, wrote:
> > > Some build dependencies are missing on www-master:
> > > 
> > > fonts-wqy-microhei fonts-vlgothic
> > 
> > Could you apply the attached patch on www-master, so that we way
> > more easily catch the issue?
> 
> Ping?

I'm working on a different patch to get mail notifications whenever the
build fails.

However, seeing the cron.git repository, I think you wanted debian-www?

> > And also please run
> > 
> > touch -t 20170101 /srv/www.debian.org/installmanual/stretch.log 
> > 
> > so that lessoften triggers the build again.
> 
> That should now get it to succeed, I have tested with the amd64 arch.
> Could some doc people do this one at last?

I've poked #debian-www to see whether it might be a good idea to get a
debwww sudo a bit wider than just update-part on devel/debian-installer
for me.


KiBi.


signature.asc
Description: Digital signature

Bug#851241: www.debian.org: packages.debian.org update hangs every now and then

[Cyril Brulebois]

Control: tag -1 patch pending

Hi,

Rhonda D'Vine  (2017-01-13):
>  I got notified today that the packages website didn't update, and that
> this seems to be an issue appearing more regularly.  I didn't had much
> time unfortunately to investigate it further, but this is what appeared
> to me:
> 
> #v+
> pkg_user 21873  0.0  0.0 103296  5192 ?SJän07   0:07 sshd: 
> pkg_user@notty
> pkg_user 21887  0.0  0.0   4336   764 ?Ss   Jän07   0:00  \_ dash -c 
> /srv/packages.debian.org/bin/push_update
> pkg_user 21898  0.0  0.0  13348  1656 ?SJän07   0:00  \_ 
> /bin/bash /srv/packages.debian.org/bin/push_update
> pkg_user 21958  0.0  0.0   4224  1476 ?SJän07   0:00  \_ 
> run-parts --verbose /srv/packages.debian.org/cron.d
> pkg_user 22309  0.0  0.0  13344  1640 ?SJän07   0:00  
> \_ /bin/bash /srv/packages.debian.org/cron.d/200proce
> pkg_user 25601 99.0  0.7 154500 122188 ?   RJän07 7804:48 
>  \_ /usr/bin/perl -w ./bin/parse-contents
> pkg_user 20378  0.0  0.0   4336   712 ?SJän07   0:00  
> \_ sh -c sort  -T /srv/packages.debian.org/tm
> pkg_user 20379  0.0  0.0   9080  3864 ?SJän07   3:24  
> \_ sort -T /srv/packages.debian.org/tmp -
> #v-
> 
>  The running time of parse-contents was increasing, but the sort didn't
> do anything. An strace showed that it was hanging:
> 
> #v+
> $ strace -p 20379
> Process 20379 attached
> write(1, "-data:kfreebsd-i386\nyp.margorp_e"..., 4096^CProcess 20379 detached
>  
> #v-
> 
>  Unfortunately I didn't had the time to investigate further, but if it
> hangs again and maybe at the same place I guess the tmp file(s) would be
> useful to investigate further.

Following a ping on #debian-devel then #debian-www I've checked what was
happening (running still Jan 15): sort was stuck on a write, sh -c stuck
on wait, and perl wasn't really doing much (no syscalls, but eating CPU,
apparently somewhere in libdb according to gdb, but I'm not sure this is
accurate / relevant).

Looking at the sizes of sid files, the aggregation is between 15-16 GB,
which might explain why piping sort's output to a perl file handle might
be suboptimal / sometimes getting stuck.

I've experimented a bit with the idea of creating a temporary file
instead, and it seems to do the job / work around this issue properly:

  
https://anonscm.debian.org/cgit/webwml/packages.git/commit/?id=1b065791a50c7a0e606f2577bcae02b7c5aa190b

I've taken the opportunity to fix progress reporting, since it can be
off by a lot with the big fat files we have in today's distributions,
through a sub called from that specific part of parse-contents, but also
from earlier in the code, cheating a bit with gzip -l's help:

  
https://anonscm.debian.org/cgit/webwml/packages.git/commit/?id=a45ab468c3d18ee671f1f65ef0cc64ebb8e9d408
  
https://anonscm.debian.org/cgit/webwml/packages.git/commit/?id=657f137df07726c41a399652618928cdc2fe8b97

Following Paul's advice, I've pushed that to master, merged into
debian-master, and deployed it on picconi since I'm still in the right
pkg group (which dates me quite a lot but can be useful at times it
seems). I'll keep an eye on cron.log, since I've killed a number of
occurrences while I was running final tests there.

Tagging with patch & pending until we get a confirmation stuff looks
better…
 

KiBi.


signature.asc
Description: Digital signature

Re: Debian Installer Stretch Alpha 6 release

[Cyril Brulebois]

Karsten Merker  (2016-05-22):
> I am not familiar with the webml syntax, so I can currently 
> only supply a wording proposal:
> 
> -8<--8<--8<--8<--8<--8<-
> 
> Wired ethernet non-functional on certain arm-based systems
> ==
> 
> Alpha 6 of the installer is based on the Linux kernel 4.5. While
> kernel 4.5 has fixed a number of issues present in earlier
> kernels, it has introduced a regression in the stmmac/dwmac
> ethernet driver.  This results in non-working wired ethernet on
> a number of systems whose ethernet controller is based on the
> stmmac/dwmac design.
> 
> The issue primarily affects arm-based systems. Currently known
> affected devices include all boards based on the Allwinner A20
> SoC (e.g. various Olimex A20-Olinuxino models, LeMaker Banana Pi
> and Banana Pro, Sinovoip Banana Pi M1, Cubietech Cubieboard2 and
> Cubietruck, LinkSprite pcDuino3).  As the stmmac/dwmac design is
> also used in other arm-based SoCs, it is expected that further
> devices will be affected as well.
> 
> Status: The issue has been fixed upstream in kernel 4.6 but the
> changes haven't yet been backported to the kernel 4.5.x stable
> series.  It is expected that the problem will be solved in Alpha
> 7 of the installer (either by the fix being backported to kernel
> 4.5 or by switching the installer to kernel 4.6).
> 
> -8<--8<--8<--8<--8<--8<-

Many thanks, I've just committed a slightly shorter version, hopefully
still accurate.

Regarding wml, that's basically html in this specific case.

FYI, the diff:
| --- errata.wml10 Jan 2016 19:08:22 -  1.221
| +++ errata.wml22 May 2016 22:16:26 -
| @@ -12,6 +12,21 @@
|  
|  
|  
| + Wired Ethernet non-functional on certain arm-based systems
| + The version 4.5 of the Linux kernel (included in the Stretch
| + Alpha 6 release) introduced a regression in the stmmac/dwmac
| + Ethernet driver
| + (https://bugs.debian.org/823493;>#823493). This
| + results in non-working wired ethernet on a number of systems
| + whose Ethernet controller is based on the stmmac/dwmac design,
| + which includes but is not limited to the following systems:
| + various Olimex A20-Olinuxino models, LeMaker Banana Pi and Banana
| + Pro, Sinovoip Banana Pi M1, Cubietech Cubieboard2 and Cubietruck,
| + LinkSprite pcDuino3.
| + 
| + Status: Will likely be fixed in the following release.
| + 
| +
|   GNOME may fail to start with some virtual machine setups
|   It was noticed during Stretch Alpha 4 image testing that
|   GNOME might fail to start depending on the setup used for virtual

Thanks again.


KiBi.


signature.asc
Description: Digital signature

Re: Bad release in install documentation

[Cyril Brulebois]

Samuel Thibault  (2016-02-06):
> Hello,
> 
> a a, on Wed 03 Feb 2016 11:03:19 +, wrote:
> > https://www.debian.org/releases/stable/i386/index.html.en. It says in
> > abstract that the documentation is for debian strech and the url says
> > about stable release.
> 
> It's worse than that: it seems it's really the current Stretch
> installation guide which ended up on the website in stable/, I don't
> know why. Www people, any idea?

Because 
http://anonscm.debian.org/cgit/debwww/cron.git/tree/lessoften-parts/1installation-guide

It would be nice if it could put stuff targetting s(-p-u) under stable, and
stuff targetting unstable under testing, but I didn't reach this point of my
todo list yet.


Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: Rebuilding translations when needed (was: translation-check's maxdelta oddities)

[Cyril Brulebois]

Hi,

Holger Wansing  (2016-01-11):
> Holger Wansing  wrote:
> > I have just committed this for the german d-i errata:
> > 
> > 
> > --- errata.wml  28 Oct 2015 20:18:01 -  1.117
> > +++ errata.wml  5 Dec 2015 22:08:26 -
> > @@ -1,6 +1,7 @@
> >  #use wml::debian::template title="Debian-Installer - Errata"
> >  #use wml::debian::recent_list
> >  #include "$(ENGLISHDIR)/devel/debian-installer/images.data"
> > +#depends "$(ENGLISHDIR)/devel/debian-installer/errata.wml"
> >  #use wml::debian::translation-check translation="1.220" mindelta="1" 
> > maxdelta="1"
> >  # $Id: errata.wml,v 1.117 2015/10/28 20:18:01 holger-guest Exp $
> >  # Original-Translator: Frank Lichtenheld , 2003-11-11
> > 
> > 
> > Let's see what happens with the next d-i alpha/beta release...
> > (and what the build log looks like tomorrow :-) German builds fine
> > here locally though.)
> 
> This did not work.
> The german page still shows 
> "Errata für Stretch Alpha 4" and no "Attention: outdated translation. Read
> the original."
> 
> Interestingly: the french site does everything right: "Alpha 5" and the
> "Attention: outdated translation. ... "
> There was no commit for french! As there was no for german.

That's what I noticed yesterday, but I didn't take time to double check
and report back.

I'm a bit lost here…

Mraw,
KiBi.


signature.asc
Description: Digital signature

Rebuilding translations when needed (was: translation-check's maxdelta oddities)

[Cyril Brulebois]

Hi,

[ Initially a debian-boot vs. debian-www topic but I think
vote.debian.org might be having a similar issue. ]

Holger Wansing <li...@wansing-online.de> (2015-10-31):
> Cyril Brulebois <k...@debian.org> wrote:
> > Could it be that images.data's being updated doesn't lead to a rebuild
> > of the relevant translations? And that those are only rebuilt when their
> > files actually get updated? That would explain why building them locally
> > led to the expected results, while the website getting updated would
> > still have the old contents?
> > 
> > (I'm not sure how dependencies between files are declared/detected, and
> > I don't think I'll find time tonight to check this out, hence just
> > throwing the idea for the time being.)
> 
> Hmm, looking at the swedish variant of that page, it seems that Kibi's
> assumption might be correct:
> 
> The corresponding swedish wml file correctly contains the entity 
> , but the website shows "Stretch Alpha 3".
> 
> There was no commit for the swedish errata file since the release of
> Alpha 4, and it seems the swedish translation of errata was not newly
> build since then.
> 
> I have no clue what to do about this though...

There's some fun going on with votes right now too.

https://www.debian.org/vote/index.fr.html has:
| En attente de sponsors
| Résolution générale : mise à jour de la procédure standard de Résolution

(i.e. waiting for seconds)

while https://www.debian.org/vote/index.en.html has:
| Voting Open
| General Resolution: Update Standard Resolution Procedure

The wml_p1_ipp manpage makes me think the following construct might help
get translations rebuilt even if their actual source didn't change:
|Special `Depends' Variant
|You can easily write fragments of Makefiles with the -M flag (see 
below) to keep tracks of which
|files the output file depends on, When "ipp" is invoked as a piece of 
"WML", the final output file
|may depend on other files.  You can tell "ipp" about these hidden 
dependencies by using the
|"#depends" variant , e.g.
| 
|  #depends 'foo.dat'
|  #depends "*/*.dat"
|  #depends 
| 
|The contents of the file is not inserted, only information about 
dependencies are updated.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: translation-check's maxdelta oddities

[Cyril Brulebois]

Hi Holger,

Holger Wansing  (2015-10-28):
> The "Stretch Alpha 3" above is an entity, which is defined in
> ../english/devel/debian-installer/images.data (which you have
> adjusted yourself :-) ).

(Updating… I do remember :))

Hmm, right. I failed to connect some dots here.

> So this means, the website was not completely newly build after you
> committed the images.data. 

Could it be that images.data's being updated doesn't lead to a rebuild
of the relevant translations? And that those are only rebuilt when their
files actually get updated? That would explain why building them locally
led to the expected results, while the website getting updated would
still have the old contents?

(I'm not sure how dependencies between files are declared/detected, and
I don't think I'll find time tonight to check this out, hence just
throwing the idea for the time being.)

> Or some kind of proxy server or cache issue.

I'm behind no proxy server, and while I didn't explicitly check with
some low-level tools like wget, I'm pretty confident I had never checked
errata in Italian before, so likely not a browser caching issue.

Thanks for your input!

Mraw,
KiBi.


signature.asc
Description: Digital signature

translation-check's maxdelta oddities

[Cyril Brulebois]

Hi,

A while ago we introduced the maxdelta parameter in translations for
some devel/debian-installer files, mainly because errata would point to
those of the previous release instead of the one which is getting
announced (until the translation is refreshed).

Looking at the live website, I don't see the expected warning, e.g. on:
  https://www.debian.org/devel/debian-installer/errata.fr.html
  (which has: “Errata de la version Stretch Alpha 3”)

or:
  https://www.debian.org/devel/debian-installer/errata.it.html
  (which has: “Errata per Stretch Alpha 3”)

Yet, building locally leads to the expected warning in
webwml/french/devel/debian-installer/errata.fr.html:
   Attention: cette traduction n'est 
plus à jour, veuillez consulter le document 
original. 

and in webwml/italian/devel/debian-installer/errata.it.html:
   Attenzione! Questa traduzione è ormai 
datata, perciò dai un'occhiata anche alla nuova pagina originale. 

Does anyone know what could explain this difference, and how to fix?

For reference: english/template/debian/translation-check.wml contains
the macro definition.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: Debian 8.2 (Jessie) has no cdimage

[Cyril Brulebois]

Hi,

Roger Shimizu  (2015-09-07):
> Debian 8.2 was released last week. But it seems that 8.2 cdimages are still
> missing in folder:
> http://cdimage.debian.org/cdimage/release/

8.2.0 is now here.

> Therefore all URLs in "Official CD/DVD images of the stable release"
> section of https://www.debian.org/CD/http-ftp/ page are dead link.

They are currently pointing at 8.1.0, which presumably got archived by
now (http://cdimage.debian.org/cdimage/archive/).

> Hope here is the right place to report this issue.

I've cc-ed debian-www@ so that they can make sure the correct version
number is used for the next website build/update.

Maybe using the “current” symlink would make sure the website still
points at the old release until the new one replaces it, without any
404s between images being published and website being updated? Of course
that means pointing at old images for a while, but the release announce
mentions that images are getting built…

Mraw,
KiBi.


signature.asc
Description: Digital signature

changelog:foo URLs don't work with '+'

[Cyril Brulebois]

Hi,

Trying to use either of those:
  https://packages.debian.org/changelog:gtk+3.0
  https://packages.debian.org/changelog:gtk%2B3.0

leads to a redirect to:
  https://packages.debian.org/changelogs/

I suppose the rewriting rules are missing something to deal with '+',
encoded or not.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: debian 8.0 errata (Re: Bug#765577: netboot install writes duplicates to 70-persistent-net.rules)

[Cyril Brulebois]

Hi!

Michael Biebl bi...@debian.org (2015-04-23):
 I talked to the release team and they prefer to postpone a fix to 8.1.
 I therefor would like to see a short paragraph added to the d-i 8.0
 errata [1].
 
 Afaics, the issue so far only happened for automated installations.
 There is no real solution/workaround ttbomk besides rebuilding d-i with
 a fixed udev-udeb. What people can do, is check
 /etc/udev/rules.d/70-persistent-net.rules and clean that up manually.
 
 
 Does the following paragraph sound ok
 
 Errata for release 8.0
 
 netboot install writes duplicate entries to 70-persistent-net.rules
 ~~~

It might be better not to embed the “netboot install” information here, in case
some other installation methods exhibit the same issue. Maybe something like
“Installation process might write […]”?

 When doing automated installations, the installer may detect NICs twice,
 Ditto. Maybe “In some situations”?

 causing network interfaces to not be configured properly. This will
 produce a shift in the names of the NICs (so e.g. eth0/eth1 may be named
 as eth1/eth2). This is caused by a udev bug (#765577) and can be
 corrected in the installed system by manually editing
 /etc/udev/rules.d/70-persistent-net.rules and /etc/network/interfaces
 accordingly.
 
 Status: Will be fixed in 8.1

This looks good to me; I suspect it would make sense to have it under both
english/devel/debian-installer/errata.wml and english/releases/jessie/errata.wml

Does debian-www@ has any clues on how to make it easy for translators to figure
out both contents are very similar, so that we avoid possible duplication of
translation efforts?

Mraw,
KiBi.

 CCed the mail in full, so KiBi has some context.
 
 [1] https://www.debian.org/releases/stable/debian-installer/
 
 Am 22.04.2015 um 14:25 schrieb Faidon Liambotis:
  reopen 765577 !
  found 765577 215-14
  thanks
  
  On Mon, Mar 30, 2015 at 06:06:47AM +0200, Marco d'Itri wrote:
  I see that we have independently devised the same fix, I am attaching 
  a test case and a more refined version of your patch.
  
  I tried Jessie RC3 today and immediately found that the fix is,
  unfortunately, buggy. Your patch constructs a regexp and takes care to
  escape metacharacters ? and * with a sed but does not escape { and
  } that are also metacharacters in the extended set of POSIX regexps.
  These are always found in the string-to-be-matched here with
  'ATTR{dev_id}==0x0' and 'ATTR{type}==1', so the if always fails.
  
  This was likely not caught by your test case (and was harder to debug
  and figure out!) because GNU grep's -E mode handles { as both a literal
  and a metacharacter heuristically for historic reasons (consult grep's
  manpage for that) but busybox grep does not:
$ echo 'foo{bar}'  test
$ egrep 'foo{bar}' test 
foo{bar}
$ busybox egrep 'foo{bar}' test 
egrep: bad regex 'foo{bar}'
$ egrep 'fo{1,2}' test 
foo{bar}
$ busybox egrep 'fo{1,2}' test 
foo{bar}
  Note that this is NOT a bug in busybox; foo{bar} is indeed an invalid
  extended POSIX regexp and busybox is right to complain and error out.
  
  The very minimal last-minute fix below did the trick for me but I have
  to say... constructing regexps in shell is tricky and the whole
  escaping-with-sed logic feels like a hack. I think a literal grep (i.e.
  -F) would be better here, especially since I don't see the point of an
  exact match (even if the file was modified by the sysadmin, the right
  thing would to not write a new rule anyway). This is probably something
  to be considered post-jessie.
  
  Thanks,
  Faidon
  
  diff --git a/debian/extra/write_net_rules b/debian/extra/write_net_rules
  index 38a3ca0..fedc0f1 100644
  --- a/debian/extra/write_net_rules
  +++ b/debian/extra/write_net_rules
  @@ -118,7 +118,7 @@ basename=${INTERFACE%%[0-9]*}
   match=$match, KERNEL==\$basename*\
   
   # build a regular expression that matches the new rule that we want to 
  write
  -new_rule_pattern=$(echo ^SUBSYSTEM==\net\, ACTION==\add\$match | sed 
  -re 's/([\?\*])/\\\1/g')
  +new_rule_pattern=$(echo ^SUBSYSTEM==\net\, ACTION==\add\$match | sed 
  -re 's/([\?\*\{\}])/\\\1/g')
   
   # Double check if the new rule has already been written. This happens if
   # multiple add events are generated before the script returns and udevd
  
  ___
  Pkg-systemd-maintainers mailing list
  pkg-systemd-maintain...@lists.alioth.debian.org
  http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-systemd-maintainers
  
 
 
 -- 
 Why is it that all of the instruments seeking intelligent life in the
 universe are pointed away from Earth?


signature.asc
Description: Digital signature

Re: Bug#765577: debian 8.0 errata (Re: Bug#765577: netboot install writes duplicates to 70-persistent-net.rules)

[Cyril Brulebois]

Michael Biebl bi...@debian.org (2015-04-23):
 Am 23.04.2015 um 22:22 schrieb Cyril Brulebois:
  Michael Biebl bi...@debian.org (2015-04-23):
 
  netboot install writes duplicate entries to 70-persistent-net.rules
  ~~~
  
  It might be better not to embed the “netboot install” information here, in 
  case
  some other installation methods exhibit the same issue. Maybe something like
  “Installation process might write […]”?
 
 Fine with me. I mostly copied the bug description here.
 
  When doing automated installations, the installer may detect NICs twice,
   Ditto. Maybe “In some situations”?
 
 I wasn't able to reproduce the bug when doing a regular, manual
 installation. That said, I have no objections being more vague here,
 just didn't want to cause unnecessary paranoia.
 
 We could do a compromise here:
 
 In some situations, the installer may detect NICs twice,
 ...
 This is caused by a udev bug (#765577) and so far has been been reported
 only for automated installations. It can be corrected ...

Looks great to me, thanks.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Partial website rebuild?

[Cyril Brulebois]

Hi people,

any chance somebody's going to be available in the upcoming (6) hours to
trigger a partial rebuild of the website (devel/debian-installer)? Just
so that I know which filename to pick (includes a date, so either today
or tomorrow).

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: Partial website rebuild?

[Cyril Brulebois]

Cyril Brulebois k...@debian.org (2015-01-25):
 any chance somebody's going to be available in the upcoming (6) hours to
 trigger a partial rebuild of the website (devel/debian-installer)? Just
 so that I know which filename to pick (includes a date, so either today
 or tomorrow).

Nevermind, I've decided to go for tomorrow and committed accordingly.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Insufficient indexing of udebs on packages.debian.org

[Cyril Brulebois]

Hi,

((
Following the instructions at the bottom of [1], and reporting on list
rather than in the BTS even if it feels wrong…

 1. https://packages.debian.org/sid/sata-modules-3.16-3-amd64-di
))

Anyway: I don't know how the “Files” column is generated but we have
“no current information” at the moment for udebs, and files present in
udebs aren't mentioned in search results.

For example, sata_mv.ko (as a filename) returns only linux-image-* [2]
matches, which are deb packages, while the same filename is also found
in the sata-modules-* udeb packages.

 2. 
https://packages.debian.org/search?searchon=contentskeywords=sata_mv.komode=exactfilenamesuite=unstablearch=any


I think the Contents files for the debian-installer subsection might be
empty, and I'm going to investigate that now, possibly filing a bug
against dak if that's indeed the case. If you're using those files, pdo
should get fixed automatically then, but if you're using something else,
an extra fix might be needed.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: Insufficient indexing of udebs on packages.debian.org

[Cyril Brulebois]

Cyril Brulebois k...@debian.org (2014-10-22):
 Anyway: I don't know how the “Files” column is generated but we have
 “no current information” at the moment for udebs, and files present in
 udebs aren't mentioned in search results.
[…]
 I think the Contents files for the debian-installer subsection might be
 empty, and I'm going to investigate that now, possibly filing a bug
 against dak if that's indeed the case. If you're using those files, pdo
 should get fixed automatically then, but if you're using something else,
 an extra fix might be needed.

OK, I was probably conflating this with some issues I had with various
files when I was mirroring using debmirror (before switching to ftpsync,
which should follow possibly incompatible changes on the archive side).


Looking at my local mirror it appears these files are available for each
suite:
  Contents-$arch.gz → symlinks to main/Contents-$arch.gz

then for i in each of main, contrib, and non-free we have:
  $i/Contents-source.gz
  $i/Contents-$arch.gz
  $i/Contents-udeb-$arch.gz

so it would probably be just a matter of indexing files matching the
last pattern to get proper udeb support if Contents files are indeed
what packages.debian.org uses.

(I'll now look into apt-file which likely needs some patching based on
the findings mentioned above, but that's no longer a topic for -www@.)

Thanks for reading so far.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: Insufficient indexing of udebs on packages.debian.org

[Cyril Brulebois]

Cyril Brulebois k...@debian.org (2014-10-22):
 Looking at my local mirror it appears these files are available for each
 suite:
   Contents-$arch.gz → symlinks to main/Contents-$arch.gz
 
 then for i in each of main, contrib, and non-free we have:
   $i/Contents-source.gz
   $i/Contents-$arch.gz
   $i/Contents-udeb-$arch.gz
 
 so it would probably be just a matter of indexing files matching the
 last pattern to get proper udeb support if Contents files are indeed
 what packages.debian.org uses.

Looking at git://anonscm.debian.org/webwml/packages.git's debian-master
branch, it might just be a matter of looping over filenames as well,
using both Contents-$arch.gz and Contents-udeb-$arch.gz (of course only
when $arch ne 'source'), in bin/parse-contents?

 (I'll now look into apt-file which likely needs some patching based on
 the findings mentioned above, but that's no longer a topic for -www@.)

(That's now #766295.)

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: Bug#763005: installation-guide-amd64: website for wheezy leads to jessie installation guide

[Cyril Brulebois]

David Prévot taf...@debian.org (2014-09-28):
 Right. There is currently nothing magic to handle what release the
 installation-guide is targeted to, so webmaster@d.o has to update one
 cron script to install it where it belongs when the release changes. If
 there is an in-advance notice before the upload, that’s smooth. If not,
 one has to dig in, and fix the stuff that had been broken (of course, it
 takes more time if nobody sends a notice at all).
 
 The Jessie manual should be rebuilt and available tomorrow where it belongs.
 
 Feel free to send patches to avoid the manual handling that has to be
 done every two years, and as such, is well expected to be often forgotten:
 
 http://anonscm.debian.org/cgit/debwww/cron.git/tree/lessoften-parts/1installation-guide

There are several places where you can find the release name:

kibi@arya:~/debian-installer/manual$ ack-grep jessie
build/buildone.sh
59:manual_release=jessie

build/build.sh
8:manual_release=${manual_release:=jessie}

build/buildweb.sh
8:manual_release=${manual_release:=jessie}

build/entities/common.ent
14:!ENTITY releasename jessie


I think we tend to update those all at once, but I think I'd go for
common.ent

 (And yes CVS-haters, this one is on Git ;-).

Hah. :)

Busy with d-i things right now, so won't be sending a patch. Maybe later
if nobody beats me to it.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Bug#747290: Please display the new Code of Conduct

[Cyril Brulebois]

Neil McGovern ne...@debian.org (2014-05-07):
 comments/corrections welcome.

Non-native speaker mode engaged.

 pDebian, the producers of the Debian system, have adopted […]

Looks very odd to me. “The Debian Project” maybe?

   listrongAssume good faith/strong
   p
 Debian Contributors have many ways of reaching our common goal of a
 a href=https://www.debian.org/intro/free;free/a operating system 
 which
 may differ from your ways. Assume that other people are working towards
 this goal.
   /pp
 Note that many of our Contributors are not native English speakers or
 may have different cultural backgrounds

Missing final dot.

[…]

 h2 id=guiidelinesFurther reading/h2

-i :)

Mraw,
KiBi.


signature.asc
Description: Digital signature

Bug#734507: wml: forgets to remove temporary /tmp/ipp.* directories

[Cyril Brulebois]

Package: wml
Version: 2.0.12ds1-3.1
Severity: important
Tags: patch

Hi,

a temp directory (/tmp/ipp.*) is created to hold a few temp files, which
are then deleted, but the container never is. This results in some
inode-eating contest, at least on the host building the Debian website.
So remove this directory on successful exit (maybe error cases would be
nice to investigate, though…).

Patch attached; also, die, 3.0 (quilt), die, but that's another story.

Mraw,
KiBi.
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,12 @@
+wml (2.0.12ds1-3.1) UNRELEASED; urgency=low
+
+  * Make sure to remove /tmp/ipp.* directories that are created using
+mkdtemp() to store temporary files, otherwise they would end up
+eating all inodes (nom nom), which is especially a pain when building
+the whole Debian website (#debian-www report by taffit).
+
+ -- Cyril Brulebois k...@debian.org  Tue, 07 Jan 2014 19:24:34 +0100
+
 wml (2.0.12ds1-3) unstable; urgency=low
 
   * Fix FTBFS in testsuite on i386 and s390x buildds by not running those
--- a/wml_backend/p1_ipp/ipp.src
+++ b/wml_backend/p1_ipp/ipp.src
@@ -682,6 +682,7 @@ else {
 }
 
 #   die gracefully
+rmdir($tmpdir);
 exit(0);
 
 ##EOF##

Re: Debian Installer 7.0 RC2 release

[Cyril Brulebois]

Cyril Brulebois k...@debian.org (27/04/2013):
 The Debian Installer team[1] is pleased to announce the second release
 candidate of the installer for Debian 7.0 Wheezy.

[…]

 Known issues in this release
 

[…]

  * Installation process might fail to detect Windows 8, on UEFI
systems (#698914).
 
 See the errata[2] for details and a full list of known issues.

Hi there.

Steve: please provide a summary + details for the Windows 8 + UEFI
issue, so that it can be added to errata, either by me or www
folks. Feel free to commit if you already have access, too.

Also, please tell us whether to keep the current “Potential issues
with UEFI booting on amd64” entry.

WWW people, please trigger a rebuild if possible (my usual mates
didn't answer on IRC this time ;)). Worst case, the website will be
updated through its next cron job.

Thanks.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: w.d.o/devel/debian-installer links and descriptions (was: w.d.o/devel/debian-installer references daily images but doesn't contain any…)

[Cyril Brulebois]

Hello David,

David Prévot da...@tilapin.org (07/11/2012):
 [ Please keep me or debian-www CC, I'm not following this list closely ]

[ Done. ]

 That has been reverted, but an actual description of the daily
 images would be welcome anyway [1]. Here is the current commented
 one, feel free to improve it and commit it uncommented:
 
  If you prefer to use the latest and greatest, either to help us
  test a future release of the installer or because of hardware
  problems or other issues, try one of these *daily built images*
  which contain the latest available version of installer
  components.

Maybe “you may try” or “you may want to try”?

Before uncommenting this section anyway, I'm wondering whether it
would make sense to add some intermediate headlines between “official
release” (currently beta4) and “current weekly snapshot”, to make it
easier to find out what each image is about.

Then we could add the same kind of section after those two, for
“current daily snapshot”.

 Furthermore, now the businesscard images vanished, would it make
 sense to place the “other images” links next to “netinst” ones in
 the first line (or would another link make more sense)? It'll keep
 CD and DVD in the same raw for the further links, just not sure we
 want “other images” to be in the first raw, but as is, it looks
 weird to have the first line half empty. Would be nice to have this
 question answered before Wheezy get published [2] (yes of course we
 have way more important stuff to do beside those nip-ticks, but
(nip/tuck?)
 we'll have way more important stuff to do than answering those
 questions when we'll be in the actual rush).

Yes, the current situation is awkward/suboptimal, but I have the same
doubts about pushing “other images” to the top. It's been a while
since I scratched the businesscard images section, but I still see no
good solutions.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: w.d.o/devel/debian-installer references daily images but doesn't contain any

[Cyril Brulebois]

David Prévot taf...@debian.org (01/11/2012):
 There was also a commented description paragraph (hidden in the HTML
 file), I just freed it, it will show up online in a few hours,
 thanks for your report.

Please revert until it is discussed on -boot.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: bts.turmzimmer.net not updating

[Cyril Brulebois]

Touko Korpela touko.korp...@iki.fi (02/06/2012):
 At least http://release.debian.org/ links to it.

Commented out from index.html for now, thanks.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Re: armhf missing on http://www.debian.org/devel/debian-installer/

[Cyril Brulebois]

Holger Levsen hol...@layer-acht.org (28/05/2012):
 armhf is completly missing on http://www.debian.org/devel/debian-installer/ 
 while eg
 http://ftp.nl.debian.org/debian/dists/testing/main/installer-armhf/current/images/
  
 does exist, just like the rest. 
 
 Makes testing the efika support kind of hard :)

One needs to find out where the arch list is defined. It's apparently
not in images.data (under webwml/english/devel/debian-installer).

Mraw,
KiBi.


signature.asc
Description: Digital signature

Bug#657557: Fix for #657557

[Cyril Brulebois]

Cyril Brulebois k...@debian.org (21/02/2012):
 If troubles arise from applying those patches, I could be tricked
 into helping to fix it up.

Experimental was only working for packages which had long descriptions
before the archive got changed, I fixed it up with the following patch.

The local diff is applied on powell and I ran a refresh to make sure it
was doing fine.

http://packages-powell.debian.org/en/experimental/libxkbcommon0

We probably want to clean up what's showing up in git status at some
point, I guess.

Mraw,
KiBi.
From 66424cad1847adc16cf339ba64ac1754ddf119a9 Mon Sep 17 00:00:00 2001
From: Cyril Brulebois k...@debian.org
Date: Fri, 2 Mar 2012 13:28:22 +
Subject: [PATCH] Bug#657557: Process translations in experimental too.

Otherwise we only get to see long descriptions and translations for
packages which were in experimental before long descriptions got
dropped from Packages files.
---
 bin/parse-translations |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/bin/parse-translations b/bin/parse-translations
index 00611a9..864173e 100755
--- a/bin/parse-translations
+++ b/bin/parse-translations
@@ -59,7 +59,7 @@ my $fixja = Text::Iconv-new(EUC-JP, UTF-8);
 # FIXME: one database per dist
 # http://lists.debian.org/4e42e104.90...@deb-support.de
 # FIXME: unhardcode dists name
-my @dists = ('sid', 'wheezy', 'squeeze', 'lenny');
+my @dists = ('experimental', 'sid', 'wheezy', 'squeeze', 'lenny');
 
 foreach my $lang (@langs) {
 (my $locale = $lang) =~ s/^([a-z]{2})-([a-z]{2})$/$1_.uc($2)/e;
-- 
1.7.2.5



signature.asc
Description: Digital signature

Bug#657557: [PATCH] Alternate patch for missing long descriptions

[Cyril Brulebois]

Goswin von Brederlow goswin-...@web.de (22/02/2012):
 Cyril and I disagree about the cause for the missing description and
 the fix for it.

Wrong, again. Please stop harrassing me, pretty fucking please.

KiBi.


signature.asc
Description: Digital signature

Bug#657557: [PATCH] Alternate patch for missing long descriptions

[Cyril Brulebois]

For the sake of getting my fellow packages/translations maintainers to
get the whole picture since you insist so much on proposing a wrong
solution and calling mine “papering over the bug”, I'll reply anyway.

Goswin von Brederlow goswin-...@web.de (22/02/2012):
 Now my patch (attached) fixes this problem by only computing the
 description-md5 field if it is missing in Packages.bz2. A simple one
 line fix. The rest of the code already does all the right things and
 looks up the translation correclty including falling back to 'en' as
 needed.

That's only if you're interesting in getting the translations back. Now
go read the block of code right above $data{'description-md5'}…

 Correct me if I'm wrong but here is how I understand Cyrils patch: It
 works by fixing the symptom instead of the problem. In [PATCH 2/4] it
 checks if the Packages.bz2 file contains an description-md5 field, If
 so it looks up the english translation for the package and replaces
 the description with the english translation, thereby restoring the
 long description for the package (line 146 with his patch).

… and this is needed so that storing the description in the database
(what I pointed to above: $descr, $sdescr, etc.) happens properly,
meaning: the long description, not the short one only.

 And now that the long description has been restored the computation of
 description-md5 a few lines later computes to the right value, the one
 that is already present.

As said on IRC, if applied after my patches, you're optimizing out an
MD5 sum in case it's present already; which doesn't hurt. But that
doesn't fix the issue with short descriptions floating around.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Bug#657557: packages.debian.org: Missing long descriptions

[Cyril Brulebois]

tag 657557 patch pending
thanks

Michał Kułach michalkul...@gmail.com (27/01/2012):
 Package: www.debian.org
 Severity: serious
 
 All long descriptions from testing, unstable and experimental are
 missing from packages.debian.org.

I think my patches should solve that. I'll send them as follow-ups to
this mail through git send-email.

 Sorry if I overestimate severity.

As you might have seen, Simon tagged the bug “confirmed”, meaning you
didn't. ;)

Mraw,
KiBi.


signature.asc
Description: Digital signature

Bug#657557: Fix for #657557

[Cyril Brulebois]

***
*** DISCLAIMER:
***   I don't run a whole instance, I only played with a few
***   languages and a few suites, checking I could perform
***   packages → long descriptions (and translations) lookups
***   again once the patches applied, and run-parts done.
***


Tiny walkthrough for those patches against debian-master:

[PATCH 1/4] Download English translations files.
 → Already applied on the Debian instance according to Rhonda.

[PATCH 2/4] Add support for long descriptions in suites above squeeze.
 → Restores long descriptions, but there's a translation→package
   dependency (when translations are updated) which disappears with
   the next two patches.

[PATCH 3/4] Add support for --english-only to parse-translations.
[PATCH 4/4] Extract English translations before processing packages.
 → I think the commit messages really say what they do: making
   processing works fine at once.


If troubles arise from applying those patches, I could be tricked
into helping to fix it up.

Mraw,
KiBi.



-- 
To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/1329789968-11695-1-git-send-email-k...@debian.org

Bug#657557: [PATCH 1/4] Download English translations files.

[Cyril Brulebois]

This will be required by the next commits.
---
 config.sh.sed.in |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/config.sh.sed.in b/config.sh.sed.in
index 9cd4c3f..ea9b355 100644
--- a/config.sh.sed.in
+++ b/config.sh.sed.in
@@ -41,7 +41,7 @@ search_url=/search
 # Architectures
 # FIXME: unhardcode archs and suites
 polangs=bg de fi fr hu ja nl pl ru sk sv uk zh-cn zh-tw
-ddtplangs=ca cs da de eo es eu fi fr hu it ja km ko nl pl pt pt-br ru sk sv 
uk zh zh-cn zh-tw
+ddtplangs=ca cs da de en eo es eu fi fr hu it ja km ko nl pl pt pt-br ru sk 
sv uk zh zh-cn zh-tw
 archives=us security debports backports volatile
 sections=main contrib non-free
 parts=$sections
-- 
1.7.2.5




-- 
To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/1329789968-11695-2-git-send-email-k...@debian.org

Bug#657557: [PATCH 2/4] Add support for long descriptions in suites above squeeze.

[Cyril Brulebois]

Use description-md5 from English translations (when available) to
restore the long descriptions during package processing.

Fix for:
  http://bugs.debian.org/657557

Archive change:
  http://lists.debian.org/debian-devel-announce/2012/01/msg4.html
---
 bin/parse-packages |   23 +++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/bin/parse-packages b/bin/parse-packages
index a1c8d98..b926982 100755
--- a/bin/parse-packages
+++ b/bin/parse-packages
@@ -59,6 +59,10 @@ $/ = ;
 -d $DBDIR/xapian.old  rmtree($DBDIR/xapian.old);
 mkpath( $DBDIR/xapian.new );
 
+# Needed to compensate removal of long descriptions from Packages files:
+my %descriptions_translated_db;
+tie %descriptions_translated_db, DB_File, 
files/db/descriptions_translated.db, O_RDONLY, 0666, $DB_BTREE;
+
 for my $suite (@SUITES) {
 my %package_names_suite = ();
 my %packages_all_db;
@@ -129,6 +133,23 @@ for my $suite (@SUITES) {
$data{'tag'} = join , , @tags;
}
 
+   # If description-md5 is present, use a lookup, thanks
+   # to the English translation which got processed right
+   # before:
+   if ($data{'description-md5'}) {
+   # The short description is a nice fallback:
+   my $description = $data{'description'};
+   my $lookup = 
$descriptions_translated_db{$data{'description-md5'}};
+   if ($lookup) {
+   while ($lookup =~ /([^\001]*)\001([^\000]*)\000/g) {
+   my ($language, $translated_description) = ($1, $2);
+   $description = $translated_description
+   if $language eq 'en';
+   }
+   $data{'description'} = $description;
+   }
+   }
+
# we add some additional data here
my $descr = $data{'description'}\000$data{'package'}\000
.($data{'tag'}||'');
@@ -194,6 +215,8 @@ for my $suite (@SUITES) {
 untie %packages_all_db;
 }
 
+untie %descriptions_translated_db;
+
 print Writing databases...\n;
 my %packages_small_db;
 tie %packages_small_db, DB_File, $DBDIR/packages_small.db.new,
-- 
1.7.2.5




-- 
To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/1329789968-11695-3-git-send-email-k...@debian.org

Bug#657557: [PATCH 3/4] Add support for --english-only to parse-translations.

[Cyril Brulebois]

That makes it possible to extract the long descriptions from English
translations since they got dropped from Packages files in some suites.
---
 bin/parse-translations |   19 +++
 1 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/bin/parse-translations b/bin/parse-translations
index d079b7f..00611a9 100755
--- a/bin/parse-translations
+++ b/bin/parse-translations
@@ -39,6 +39,17 @@ use Packages::Config qw( $TOPDIR $DBDIR @DDTP_LANGUAGES );
 Packages::Config::init( './' );
 my %descriptions = ();
 
+# Make it possible to deal with either only English translations (to
+# get long descriptions back), or all of them (the default). Since a
+# single option needs to be supported, don't bother with getopt:
+my @langs = @DDTP_LANGUAGES;
+my $output = 'descriptions_translated';
+my $argument = shift @ARGV;
+if ($argument and $argument eq '--english-only') {
+$output = 'descriptions_translated_english_only';
+@langs = ('en');
+}
+
 $/ = ;
 
 -d $DBDIR || mkpath( $DBDIR );
@@ -50,7 +61,7 @@ my $fixja = Text::Iconv-new(EUC-JP, UTF-8);
 # FIXME: unhardcode dists name
 my @dists = ('sid', 'wheezy', 'squeeze', 'lenny');
 
-foreach my $lang (@DDTP_LANGUAGES) {
+foreach my $lang (@langs) {
 (my $locale = $lang) =~ s/^([a-z]{2})-([a-z]{2})$/$1_.uc($2)/e;
 print Reading Translations for $lang ($locale)...;
 my $count = 0;
@@ -87,7 +98,7 @@ close PKG;
 
 print Writing database (.scalar(keys %descriptions). unique 
descriptions)...\n;
 my %descriptions_db;
-tie %descriptions_db, DB_File, $DBDIR/descriptions_translated.db.new,
+tie %descriptions_db, DB_File, $DBDIR/$output.db.new,
O_RDWR|O_CREAT, 0666, $DB_BTREE
or die Error creating DB: $!;
 while (my ($md5, $v) = each(%descriptions)) {
@@ -104,5 +115,5 @@ while (my ($md5, $v) = each(%descriptions)) {
 }
 untie %descriptions_db;
 
-rename($DBDIR/descriptions_translated.db.new,
-   $DBDIR/descriptions_translated.db);
+rename($DBDIR/$output.db.new,
+   $DBDIR/$output.db);
-- 
1.7.2.5




-- 
To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/1329789968-11695-4-git-send-email-k...@debian.org

Bug#657557: [PATCH 4/4] Extract English translations before processing packages.

[Cyril Brulebois]

Since long descriptions are determined using English translations (for
some suites only, right now), it's needed to process them before
Packages files are processed.
---
 bin/parse-packages|   14 --
 cron.d/200process_archive |2 ++
 2 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/bin/parse-packages b/bin/parse-packages
index b926982..e75a500 100755
--- a/bin/parse-packages
+++ b/bin/parse-packages
@@ -60,8 +60,8 @@ $/ = ;
 mkpath( $DBDIR/xapian.new );
 
 # Needed to compensate removal of long descriptions from Packages files:
-my %descriptions_translated_db;
-tie %descriptions_translated_db, DB_File, 
files/db/descriptions_translated.db, O_RDONLY, 0666, $DB_BTREE;
+my %descriptions_english_db;
+tie %descriptions_english_db, DB_File, 
files/db/descriptions_translated_english_only.db, O_RDONLY, 0666, $DB_BTREE;
 
 for my $suite (@SUITES) {
 my %package_names_suite = ();
@@ -139,11 +139,13 @@ for my $suite (@SUITES) {
if ($data{'description-md5'}) {
# The short description is a nice fallback:
my $description = $data{'description'};
-   my $lookup = 
$descriptions_translated_db{$data{'description-md5'}};
+   my $lookup = 
$descriptions_english_db{$data{'description-md5'}};
if ($lookup) {
+   # There should only be an English translation
+   # in there, but let's make sure:
while ($lookup =~ /([^\001]*)\001([^\000]*)\000/g) {
-   my ($language, $translated_description) = ($1, $2);
-   $description = $translated_description
+   my ($language, $english_description) = ($1, $2);
+   $description = $english_description
if $language eq 'en';
}
$data{'description'} = $description;
@@ -215,7 +217,7 @@ for my $suite (@SUITES) {
 untie %packages_all_db;
 }
 
-untie %descriptions_translated_db;
+untie %descriptions_english_db;
 
 print Writing databases...\n;
 my %packages_small_db;
diff --git a/cron.d/200process_archive b/cron.d/200process_archive
index b8f6a6d..29a7385 100755
--- a/cron.d/200process_archive
+++ b/cron.d/200process_archive
@@ -5,6 +5,8 @@
 cd $topdir
 
 date
+./bin/parse-translations --english-only
+date
 ./bin/parse-packages
 date
 ./bin/parse-sources
-- 
1.7.2.5




-- 
To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/1329789968-11695-5-git-send-email-k...@debian.org

Re: [packages.debian.org] Please index udebs in experimental

[Cyril Brulebois]

Simon Paillard spaill...@debian.org (11/01/2011):
 Done.
 
 http://git.debian.org/?p=webwml/packages.git;a=commit;h=1cd6b78f69f171c418386ed2d6ce4fd5e1e9570a

Thanks!

KiBi.


signature.asc
Description: Digital signature

[packages.debian.org] Please index udebs in experimental

[Cyril Brulebois]

Hi,

mailing debian-www@ per packages.debian.org's footer, even though I
thought there was some plan to add a more specific footer pointing
(directly) to the people behind this service, and cc-ing -boot@ since
that's about udebs.

Currently udebs are shown for lenny to sid, but not for
experimental. An example is libpango1.0-udeb:
  http://packages.debian.org/search?libpango1.0-udeb

Of course, the version in experimental also ships an udeb with the
same name.

Another example is xserver-xorg-core-udeb, which is only shown for
testing and sid, while the same udeb is available in experimental:
  http://packages.debian.org/search?xserver-xorg-core-udeb

It would be nice to display the udebs in experimental as well, which
would be particularly useful for those who would like to build d-i
images with some packages picked from experimental. See “Premises” for
a use case:
  http://blog.ikibiki.org/2011/01/10/Giving_d-i_rc1_a_snapshot/

Please Cc either -boot@ or me in your replies.

Thanks!

KiBi.


signature.asc
Description: Digital signature

Bug#559937: packages.debian.org: version mismatch for libcwidget3

[Cyril Brulebois]

Package: www.debian.org
Severity: normal

Hi,

I've just stumbled upon this bug:
| k...@kbsd:~$ rmadison -a kfreebsd-i386 libcwidget3
| libcwidget3 |   0.5.13-1 |   testing | kfreebsd-i386
| libcwidget3 |   0.5.16-2 |  unstable | kfreebsd-i386

while http://packages.debian.org/libcwidget3 says:
| squeeze (testing) (libs): high-level terminal interface library for C++ 
(runtime files) 
| 0.5.13-1: amd64 armel hppa i386 ia64 mips mipsel powerpc s390 sparc

dak is right here, there's indeed such a version in testing for
kfreebsd-i386:
| k...@kbsd:~$ apt-cache policy libcwidget3|grep Packages
| 500 ftp://ftp.fr.debian.org sid/main Packages
| 500 ftp://ftp.fr.debian.org squeeze/main Packages

Given the versions are different in testing and unstable, I don't think
it's due to a just-happened migration and to a lag of some sort.

Mraw,
KiBi.

(By the way, why not pointing to the appropriate pseudopackage in the
footer of packages.debian.org pages, rather than to -...@?)



-- 
To UNSUBSCRIBE, email to debian-www-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org