Your message dated Fri, 5 Jan 2024 18:33:02 +0100
with message-id
<j4c7ryfmra4fxw6rvqlmroibfqjzaurook3mbjtojwgcizrs3y@ugnmzurhltqk>
and subject line Re: Bug#1060061: grub-efi-amd64-bin: out of memory when
loading initial ramdisk
has caused the Debian Bug report #1060061,
regarding grub-efi-amd64-bin: out of memory when loading initial ramdisk
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)
--
1060061: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1060061
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: grub-efi-amd64-bin
Version: 2.06-3~deb11u6
Severity: critical
Tags: upstream
Justification: breaks the whole system
X-Debbugs-Cc: mtilje...@uber.com
Dear Maintainer,
On certain system, in this case an Ice Lake based server running an AMI
bios (version 3219BS01.021), GRUB will sporadically fail to boot with
the following:
>> Booting `Debian GNU/Linux, with Linux 5.10.113-1'Booting `Debian
>> GNU/Linux, with Linux 5.10.113-1'
>>
>>
>>
>> Loading Linux 5.10.113-1 ...Loading Linux 5.10.113-1 ...
>>
>> Loading initial ramdisk ...Loading initial ramdisk ...
>>
>> error: out of memory.
>> error: out of memory.
This error is sporadic and appears maybe 90% of the time, so I can
barely get the system up and running for testing purposes.
I believe this is related to memory management in GRUB and similar to
https://bugs.launchpad.net/oem-priority/+bug/1842320
which is apparently caused by an upstream bug
https://savannah.gnu.org/bugs/?61058
>From digging around it appears this has been fixed in Redhat/Fedora with
https://src.fedoraproject.org/rpms/grub2/c/098a8a9e99d4b32576e7af518e8e195059f985b9?branch=f33
but apparently this patch applies to previous memory subsystem patches,
so I haven't been able to backport these patches.
I've successfully backported grub 2.12~rc1-12 from sid by changing the
compiler back to gcc-10 and building from source. This version works and
boots every time without issue.
I've also tried to backport grub 2.06-13+deb12u1 from bookworm, but this
does not help. I hence believe this problem affects both Debian
bullseye and bookworm.
Best,
Morten Hein Tiljeset
-- Package-specific info:
*********************** BEGIN /proc/mounts
/dev/nvme0n1p1 / ext4 rw,noatime,errors=panic 0 0
/dev/nvme1n1p1 /opt ext4 rw,noatime,errors=panic,stripe=64 0 0
/dev/nvme1n1p1 /var/log ext4 rw,noatime,errors=panic,stripe=64 0 0
/dev/nvme0n1p12 /boot/efi vfat
rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro
0 0
/dev/nvme1n1p1 /var/lib/systemd/coredump ext4 rw,noatime,errors=panic,stripe=64
0 0
/dev/nvme1n1p1 /var/log/containers ext4 rw,noatime,errors=panic,stripe=64 0 0
/dev/nvme1n1p1 /opt/system/log/containers ext4
rw,noatime,errors=panic,stripe=64 0 0
/dev/nvme1n1p1 /var/log/pods ext4 rw,noatime,errors=panic,stripe=64 0 0
/dev/nvme1n1p1 /opt/system/log/pods ext4 rw,noatime,errors=panic,stripe=64 0 0
*********************** END /proc/mounts
*********************** BEGIN /boot/grub/grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#
### BEGIN /etc/grub.d/00_header ###
if [ -s $prefix/grubenv ]; then
set have_grubenv=true
load_env
fi
if [ "${next_entry}" ] ; then
set default="${next_entry}"
set next_entry=
save_env next_entry
set boot_once=true
else
set
default="gnulinux-5.10.113-1-advanced-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9"
fi
if [ x"${feature_menuentry_id}" = xy ]; then
menuentry_id_option="--id"
else
menuentry_id_option=""
fi
export menuentry_id_option
if [ "${prev_saved_entry}" ]; then
set saved_entry="${prev_saved_entry}"
save_env saved_entry
set prev_saved_entry=
save_env prev_saved_entry
set boot_once=true
fi
function savedefault {
if [ -z "${boot_once}" ]; then
saved_entry="${chosen}"
save_env saved_entry
fi
}
function load_video {
if [ x$feature_all_video_module = xy ]; then
insmod all_video
else
insmod efi_gop
insmod efi_uga
insmod ieee1275_fb
insmod vbe
insmod vga
insmod video_bochs
insmod video_cirrus
fi
}
serial --speed=115200
terminal_input console serial
terminal_output console serial
if [ "${recordfail}" = 1 ] ; then
set timeout=30
else
if [ x$feature_timeout_style = xy ] ; then
set timeout_style=menu
set timeout=0
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
set timeout=0
fi
fi
### END /etc/grub.d/00_header ###
### BEGIN /etc/grub.d/04_unrestricted ###
menuentry_id_option="--unrestricted $menuentry_id_option"
### END /etc/grub.d/04_unrestricted ###
### BEGIN /etc/grub.d/05_debian_theme ###
set menu_color_normal=cyan/blue
set menu_color_highlight=white/blue
### END /etc/grub.d/05_debian_theme ###
### BEGIN /etc/grub.d/05_header ###
### END /etc/grub.d/05_header ###
### BEGIN /etc/grub.d/10_linux ###
function gfxmode {
set gfxpayload="${1}"
}
set linux_gfx_mode=
export linux_gfx_mode
menuentry 'Debian GNU/Linux, with Linux 6.1.53-1' --class debian --class
gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-6.1.53-1-advanced-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 6.1.53-1 ...'
linux /boot/vmlinuz-6.1.53-1
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-6.1.53-1
}
menuentry 'Debian GNU/Linux, with Linux 6.1.53-1 (recovery mode)' --class
debian --class gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-6.1.53-1-recovery-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 6.1.53-1 ...'
linux /boot/vmlinuz-6.1.53-1
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro single
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-6.1.53-1
}
menuentry 'Debian GNU/Linux, with Linux 6.1.30-4' --class debian --class
gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-6.1.30-4-advanced-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 6.1.30-4 ...'
linux /boot/vmlinuz-6.1.30-4
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-6.1.30-4
}
menuentry 'Debian GNU/Linux, with Linux 6.1.30-4 (recovery mode)' --class
debian --class gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-6.1.30-4-recovery-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 6.1.30-4 ...'
linux /boot/vmlinuz-6.1.30-4
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro single
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-6.1.30-4
}
menuentry 'Debian GNU/Linux, with Linux 5.10.113-1' --class debian --class
gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-5.10.113-1-advanced-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 5.10.113-1 ...'
linux /boot/vmlinuz-5.10.113-1
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-5.10.113-1
}
menuentry 'Debian GNU/Linux, with Linux 5.10.113-1 (recovery mode)' --class
debian --class gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-5.10.113-1-recovery-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 5.10.113-1 ...'
linux /boot/vmlinuz-5.10.113-1
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro single
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-5.10.113-1
}
menuentry 'Debian GNU/Linux, with Linux 4.14.171-5' --class debian --class
gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-4.14.171-5-advanced-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 4.14.171-5 ...'
linux /boot/vmlinuz-4.14.171-5
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-4.14.171-5
}
menuentry 'Debian GNU/Linux, with Linux 4.14.171-5 (recovery mode)' --class
debian --class gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-4.14.171-5-recovery-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 4.14.171-5 ...'
linux /boot/vmlinuz-4.14.171-5
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro single
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-4.14.171-5
}
menuentry 'Debian GNU/Linux, with Linux 4.14.171' --class debian --class
gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-4.14.171-advanced-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 4.14.171 ...'
linux /boot/vmlinuz-4.14.171
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-4.14.171
}
menuentry 'Debian GNU/Linux, with Linux 4.14.171 (recovery mode)' --class
debian --class gnu-linux --class gnu --class os $menuentry_id_option
'gnulinux-4.14.171-recovery-5a78d20e-e0e5-44b7-b2a6-363a3b536fc9' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod ext2
search --no-floppy --fs-uuid --set=root
5a78d20e-e0e5-44b7-b2a6-363a3b536fc9
echo 'Loading Linux 4.14.171 ...'
linux /boot/vmlinuz-4.14.171
root=UUID=5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ro single
systemd.legacy_systemd_cgroup_controller=false
systemd.unified_cgroup_hierarchy=false console=tty0 console=ttyS0,115200n8
noresume mitigations=off nvme_core.default_ps_max_latency_us=0
tpm_tis.interrupts=0
echo 'Loading initial ramdisk ...'
initrd /boot/initrd.img-4.14.171
}
### END /etc/grub.d/10_linux ###
### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###
### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###
### BEGIN /etc/grub.d/30_uefi-firmware ###
### END /etc/grub.d/30_uefi-firmware ###
### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###
### BEGIN /etc/grub.d/41_custom ###
if [ -f ${config_directory}/custom.cfg ]; then
source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f $prefix/custom.cfg ]; then
source $prefix/custom.cfg
fi
### END /etc/grub.d/41_custom ###
*********************** END /boot/grub/grub.cfg
*********************** BEGIN /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4]
[raid10]
unused devices: <none>
*********************** END /proc/mdstat
*********************** BEGIN /dev/disk/by-id
total 0
lrwxrwxrwx 1 root root 13 Jan 5 12:14 nvme-KXG60ZNV512G_TOSHIBA_43TFC09RFXS4
-> ../../nvme0n1
lrwxrwxrwx 1 root root 15 Jan 5 12:14
nvme-KXG60ZNV512G_TOSHIBA_43TFC09RFXS4-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 16 Jan 5 12:14
nvme-KXG60ZNV512G_TOSHIBA_43TFC09RFXS4-part12 -> ../../nvme0n1p12
lrwxrwxrwx 1 root root 15 Jan 5 12:14
nvme-KXG60ZNV512G_TOSHIBA_43TFC09RFXS4-part2 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 13 Jan 5 12:14
nvme-Micron_7450_MTFDKCC3T8TFR_232042112042 -> ../../nvme1n1
lrwxrwxrwx 1 root root 15 Jan 5 12:14
nvme-Micron_7450_MTFDKCC3T8TFR_232042112042-part1 -> ../../nvme1n1p1
lrwxrwxrwx 1 root root 13 Jan 5 12:14
nvme-eui.000000000000000100a0752342112042 -> ../../nvme1n1
lrwxrwxrwx 1 root root 15 Jan 5 12:14
nvme-eui.000000000000000100a0752342112042-part1 -> ../../nvme1n1p1
lrwxrwxrwx 1 root root 13 Jan 5 12:14
nvme-eui.00000000000000018ce38e0500e17bb7 -> ../../nvme0n1
lrwxrwxrwx 1 root root 15 Jan 5 12:14
nvme-eui.00000000000000018ce38e0500e17bb7-part1 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 16 Jan 5 12:14
nvme-eui.00000000000000018ce38e0500e17bb7-part12 -> ../../nvme0n1p12
lrwxrwxrwx 1 root root 15 Jan 5 12:14
nvme-eui.00000000000000018ce38e0500e17bb7-part2 -> ../../nvme0n1p2
*********************** END /dev/disk/by-id
*********************** BEGIN /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 15 Jan 5 12:14 17adc89f-8a60-4383-b48f-27c9237b9ea7 ->
../../nvme1n1p1
lrwxrwxrwx 1 root root 15 Jan 5 12:14 5a78d20e-e0e5-44b7-b2a6-363a3b536fc9 ->
../../nvme0n1p1
lrwxrwxrwx 1 root root 16 Jan 5 12:14 B9B1-3121 -> ../../nvme0n1p12
*********************** END /dev/disk/by-uuid
-- System Information:
Debian Release: 11.8
APT prefers oldstable-updates
APT policy: (500, 'oldstable-updates'), (500, 'oldstable-security'), (500,
'oldstable')
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.113-1 (SMP w/96 CPU threads)
Kernel taint flags: TAINT_UNSIGNED_MODULE
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968) (ignored: LC_ALL set to C),
LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages grub-efi-amd64-bin depends on:
ii grub-common 2.06-3~deb11u6
Versions of packages grub-efi-amd64-bin recommends:
pn efibootmgr <none>
ii grub-efi-amd64-signed 1+2.06+3~deb11u6
grub-efi-amd64-bin suggests no packages.
-- no debconf information
--- End Message ---
--- Begin Message ---
Version: 2.12~rc1-1
On Fri, Jan 05, 2024 at 12:56:54PM +0000, Morten Hein Tiljeset wrote:
> Package: grub-efi-amd64-bin
> Version: 2.06-3~deb11u6
> Severity: critical
> Tags: upstream
> Justification: breaks the whole system
> X-Debbugs-Cc: mtilje...@uber.com
>
> Dear Maintainer,
>
> On certain system, in this case an Ice Lake based server running an AMI
> bios (version 3219BS01.021), GRUB will sporadically fail to boot with
> the following:
> >> Booting `Debian GNU/Linux, with Linux 5.10.113-1'Booting `Debian
> >> GNU/Linux, with Linux 5.10.113-1'
> >>
> >>
> >>
> >> Loading Linux 5.10.113-1 ...Loading Linux 5.10.113-1 ...
> >>
> >> Loading initial ramdisk ...Loading initial ramdisk ...
> >>
> >> error: out of memory.
> >> error: out of memory.
>
> This error is sporadic and appears maybe 90% of the time, so I can
> barely get the system up and running for testing purposes.
>
> I believe this is related to memory management in GRUB and similar to
> https://bugs.launchpad.net/oem-priority/+bug/1842320
> which is apparently caused by an upstream bug
> https://savannah.gnu.org/bugs/?61058
>
> From digging around it appears this has been fixed in Redhat/Fedora with
> https://src.fedoraproject.org/rpms/grub2/c/098a8a9e99d4b32576e7af518e8e195059f985b9?branch=f33
> but apparently this patch applies to previous memory subsystem patches,
> so I haven't been able to backport these patches.
>
> I've successfully backported grub 2.12~rc1-12 from sid by changing the
> compiler back to gcc-10 and building from source. This version works and
> boots every time without issue.
>
> I've also tried to backport grub 2.06-13+deb12u1 from bookworm, but this
> does not help. I hence believe this problem affects both Debian
> bullseye and bookworm.
>
I'm afraid touching the memory management system in the 2.06 grub in
stable releases is out of the question. The fixes involve a ton of
patches and it's not really possible to cherry-pick them and the risk
of regression is significant and out of scope for (only) a stable
release.
The only way we managed to in Ubuntu was to cherry-pick all the loader
changes from rhboot, and then also cherry-pick the entire 2.12 memory
management changes on top of it, and that caused plenty of regressions
at each point we picked more changes...
In my opinion, this is not a release critical bug breaking _existing_
systems, but a request to enable additional hardware currently
unsupported in debian.
We will eventually backport the 2.12 grub to stable releases in
the upcoming months. We may want to issue it via -backports at first,
but if a security vulnerability comes in, this will eventually land
in a point release.
As such I am closing this bug as fixed in 2.12~rc1. Thank you for
your understanding.
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
--- End Message ---
