Re: mdadm /boot mirror and sun disklabel corruption

2017-09-06 Thread Frank Scheiner 

On 09/06/2017 05:21 PM, Fedor Konstantinov wrote:

I'm creating mirrored system disk.
For example I make partitions on two disks like the following:
1. 500MB for /boot - boot partition
2. 2GB for swap - swap
3. Whole disk - sun's whole disk
4. 31,6GB for / - rest for the root fs

Then I create metadevices (mirrors) for partitions 1,2 and 4.

We know, that sun disk label (partition table) resides at the beginning 
of the disk. In our case partition 1 (for the /boot filesystem) is also 
at the beginning of the disk.
When debian installer creates mdadm metadata v1.2 for partition 1 it 
overwrites sun disklabel. As a result after installation OBP can't read 
disk label and boot the system.
Version 0.90 metadata resides at the end of partition, so it is safe to 
use it for partitions at the beginning of disks with sun disklabel.


Unfortunately, in the debian installer we don't have possibility to 
select metadata version during install. :(


I'm not familiar with all the details of mdadm, but the way you describe 
it, it sounds like mdadm metadata is per partition and not per disk.


Then couldn't this issue be worked around by creating a small unused 
partition at the beginning of the disk in question which hence offsets 
the partition for `/boot`? So that mdadm v1.2 metadata for the `/boot` 
partition does not end in the Sun disk label?

Re: Re: mdadm /boot mirror and sun disklabel corruption

2017-09-06 Thread Hermann Lauer 
Hey Adrian,
from the old, golden sparc days:
On Wed, Sep 06, 2017 at 04:59:44PM +0200, John Paul Adrian Glaubitz wrote:
> On 09/06/2017 04:40 PM, Fedor Konstantinov wrote:
> > I would greatly appreciate if it could be possible to add to the installer 
> > ability to
> > select mdadm metadata version when creating RAID arrays. Now, when creating 
> > mirror for
> > /boot partition (at sun disk slice 0) installer uses v1.2 metadata and so 
> > it corrupts
> > sun disklabel. To avoid disklabel corruption, now I have to manually create 
> > such partition
> > with v0.90 metadata, and ext3 fs in it.

I used v1.0 metadata for the same reason: slice 0 contains at the beginning
partitioning information, which will be overwritten by >1.0 versions. Up
to there raid bitmaps where at the end. Setup worked with breaking out to a
shell, creating raid1 and the continue in the installer.

> So, you want to use an old metadata format? Wouldn't it make more sense to 
> fix the bug
> in question?
> 
> Also, you want to create a mirrored boot directory, is that correct?

Did that - caveeat was, that the partition table is mirrored, so only
the layout of the smaller disk could be used.

Greetings
  Hermann

-- 
Netzwerkadministration/Zentrale Dienste, Interdiziplinaeres 
Zentrum fuer wissenschaftliches Rechnen der Universitaet Heidelberg
IWR; INF 205; 69120 Heidelberg; Tel: (06221)54-14405 Fax: -14427
Email: hermann.la...@iwr.uni-heidelberg.de

Re: mdadm /boot mirror and sun disklabel corruption

2017-09-06 Thread Fedor Konstantinov 

I'm creating mirrored system disk.
For example I make partitions on two disks like the following:
1. 500MB for /boot - boot partition
2. 2GB for swap - swap
3. Whole disk - sun's whole disk
4. 31,6GB for / - rest for the root fs

Then I create metadevices (mirrors) for partitions 1,2 and 4.

We know, that sun disk label (partition table) resides at the beginning 
of the disk. In our case partition 1 (for the /boot filesystem) is also 
at the beginning of the disk.
When debian installer creates mdadm metadata v1.2 for partition 1 it 
overwrites sun disklabel. As a result after installation OBP can't read 
disk label and boot the system.
Version 0.90 metadata resides at the end of partition, so it is safe to 
use it for partitions at the beginning of disks with sun disklabel.


Unfortunately, in the debian installer we don't have possibility to 
select metadata version during install. :(


06.09.2017 17:59, John Paul Adrian Glaubitz пишет:

On 09/06/2017 04:40 PM, Fedor Konstantinov wrote:
I would greatly appreciate if it could be possible to add to the 
installer ability to
select mdadm metadata version when creating RAID arrays. Now, when 
creating mirror for
/boot partition (at sun disk slice 0) installer uses v1.2 metadata 
and so it corrupts
sun disklabel. To avoid disklabel corruption, now I have to manually 
create such partition

with v0.90 metadata, and ext3 fs in it.


So, you want to use an old metadata format? Wouldn't it make more 
sense to fix the bug

in question?

Also, you want to create a mirrored boot directory, is that correct?

Adrian

Re: mdadm /boot mirror and sun disklabel corruption

2017-09-06 Thread John Paul Adrian Glaubitz 

On 09/06/2017 04:40 PM, Fedor Konstantinov wrote:

I would greatly appreciate if it could be possible to add to the installer 
ability to
select mdadm metadata version when creating RAID arrays. Now, when creating 
mirror for
/boot partition (at sun disk slice 0) installer uses v1.2 metadata and so it 
corrupts
sun disklabel. To avoid disklabel corruption, now I have to manually create 
such partition
with v0.90 metadata, and ext3 fs in it.


So, you want to use an old metadata format? Wouldn't it make more sense to fix 
the bug
in question?

Also, you want to create a mirrored boot directory, is that correct?

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

mdadm /boot mirror and sun disklabel corruption

2017-09-06 Thread Fedor Konstantinov 

Hi, list

I would greatly appreciate if it could be possible to add to the 
installer ability to select mdadm metadata version when creating RAID 
arrays.
Now, when creating mirror for /boot partition (at sun disk slice 0) 
installer uses v1.2 metadata and so it corrupts sun disklabel.
To avoid disklabel corruption, now I have to manually create such 
partition with v0.90 metadata, and ext3 fs in it.


Regards,
Fedor

Re: Fwd: Re: Latest Debian SPARC64 ISO Image

2017-09-06 Thread John Paul Adrian Glaubitz 

On 09/06/2017 03:00 PM, Frank Scheiner wrote:

I can confirm this. But at least the boot loader can now be installed from 
rescue mode as silo gets installed in the target root FS.

I tested the default installation, which went through as before without an 
issue - apart from the missing boot loader installation.


This is most likely related to silo-installer missing or not doing its job.

I will look into this.


As Adrian pointed out in [1] a separate partition with an EXT3 FS (or most 
likely also EXT2) on it for `/boot` is essential.
I first tried with an EXT4 file system on the root partition and no separate 
partition for `/boot`, but silo wasn't able
to access files there. I assume it could work with an EXT3 FS on the root 
partition, but I haven't tested this.


If your SPARC server isn't too old, you can actually also use GRUB:


https://github.com/esnowberg/grub2-sparc/wiki



So far I recognized no obvious problems with the installation.

* There is an empty `sources.list~` remaining in `/etc/apt`, but that's not a 
problem at all.


Thanks for catching this. I will try to figure out where this comes from,
but it is most likely caused by choose-mirror:


https://anonscm.debian.org/git/d-i/choose-mirror.git/



W: Skipping acquire of configured file 'main/source/Sources' as repository
'http://ftp.ports.debian.org/debian-ports sid InRelease' does not seem to 
provide it (sources.list entry misspelt?)


That's because choose-mirror tries to use the same mirror for binary and
source packages. However, there are no source packages on the Debian
Ports mirrors. Just replace the sources.list entry with a regular
mirror entry, i.e.:


deb-src http://ftp.$COUNTRY.debian.org/debian/ sid main


Where $COUNTRY is something like de, us, jp and so on.


* As I use it for my alpha, hppa and ppc64 machines, shouldn't there not also be a line 
for the "unreleased" suite?


Yes. We would need to patch choose-mirror here as well.

Feel free to have a look at the code yourself and suggest a patch.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Updated debian-installer images

2017-09-06 Thread Frank Scheiner 

On 09/06/2017 12:22 PM, James Clarke wrote:

I only made a couple of minor changes; the edit you want is [0] by Alex
McWhirter.

James

[0] https://wiki.debian.org/Sparc64?action=diff&rev1=25&rev2=26


Thanks for the pointer, I'll contact him directly.

Re: Fwd: Re: Latest Debian SPARC64 ISO Image

2017-09-06 Thread Frank Scheiner 

On 09/06/2017 12:36 PM, Fedor Konstantinov wrote:

Hi,

there's no "Install SILO boot loader" option in the installer menu in 
the latest (06.09.2017) debian-9.0-sparc64-NETINST-1.iso image. Despite 
of that SILO is installed.


Regards,
Fedor


I can confirm this. But at least the boot loader can now be installed 
from rescue mode as silo gets installed in the target root FS.


I tested the default installation, which went through as before without 
an issue - apart from the missing boot loader installation.


As Adrian pointed out in [1] a separate partition with an EXT3 FS (or 
most likely also EXT2) on it for `/boot` is essential. I first tried 
with an EXT4 file system on the root partition and no separate partition 
for `/boot`, but silo wasn't able to access files there. I assume it 
could work with an EXT3 FS on the root partition, but I haven't tested this.


[1]: https://lists.debian.org/debian-sparc/2016/06/msg00126.html

After manually installing the silo boot loader, the V245 could be booted 
without an issue:

```
Sun Fire V245, No Keyboard
Copyright 2007 Sun Microsystems, Inc.  All rights reserved.
OpenBoot 4.25.10, 12288 MB memory installed, Serial #11223344.
Ethernet address 0:14:4f:11:22:33, Host ID: 84112233.



Rebooting with command: boot disk0
Boot device: /pci@1e,60/pci@0/pci@a/pci@0/pci@8/scsi@1/disk@0,0 
File and args:

SILO Version 1.4.14
boot:
Allocated 64 Megs of memory at 0x4000 for kernel
Uncompressing image...
Loaded kernel version 4.12.6
Loading initial ramdisk (17937980 bytes at 0x120400 phys, 0x40C0 
virt)...

/
[0.00] PROMLIB: Sun IEEE Boot Prom 'OBP 4.25.10 2007/09/18 09:56'
[0.00] PROMLIB: Root node compatible:
[0.00] Linux version 4.12.0-1-sparc64-smp 
(debian-ker...@lists.debian.org) (gcc version 6.4.0 20170805 (Debian 
6.4.0-3) ) #1 SMP Debian 4.12.6-1 (2017-08-12)

[...]
[   90.955416] systemd[1]: systemd 234 running in system mode. (+PAM 
+AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP 
+GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN 
default-hierarchy=hybrid)

[   91.228427] systemd[1]: Detected architecture sparc64.

Welcome to Debian GNU/Linux buster/sid!
[...]
Debian GNU/Linux buster/sid v245 ttyS0

v245 login: root
Password:
Linux v245 4.12.0-1-sparc64-smp #1 SMP Debian 4.12.6-1 (2017-08-12) sparc64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
root@v245:~#
```

So far I recognized no obvious problems with the installation.

* There is an empty `sources.list~` remaining in `/etc/apt`, but that's 
not a problem at all.


* And `apt` complains about a possibly misspelt sources.list entry:
```
root@v245:/etc/apt# apt update
Hit:1 http://ftp.ports.debian.org/debian-ports sid InRelease
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.
W: Skipping acquire of configured file 'main/source/Sources' as 
repository 'http://ftp.ports.debian.org/debian-ports sid InRelease' does 
not seem to provide it (sources.list entry misspelt?)

root@v245:/etc/apt# cat /etc/apt/sources.list
#

# deb cdrom:[Debian GNU/Linux 9.0 _Sid_ - Unofficial sparc64 NETINST 
20170905-22:45]/ sid main


#deb cdrom:[Debian GNU/Linux 9.0 _Sid_ - Unofficial sparc64 NETINST 
20170905-22:45]/ sid main


deb http://ftp.ports.debian.org/debian-ports/ sid main
deb-src http://ftp.ports.debian.org/debian-ports/ sid main
```
Also no issue for me.

* As I use it for my alpha, hppa and ppc64 machines, shouldn't there not 
also be a line for the "unreleased" suite?

Re: Fwd: Re: Latest Debian SPARC64 ISO Image

2017-09-06 Thread John Paul Adrian Glaubitz 

On 09/06/2017 12:36 PM, Fedor Konstantinov wrote:

there's no "Install SILO boot loader" option in the installer menu in the
latest (06.09.2017) debian-9.0-sparc64-NETINST-1.iso image. Despite of that
SILO is installed.

Thanks for the heads-up. I will look into this issue. I'm still ironing
out some remaining bugs in the CD image creation process.

Adrian

--
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Fwd: Re: Latest Debian SPARC64 ISO Image

2017-09-06 Thread Fedor Konstantinov 

Hi,

there's no "Install SILO boot loader" option in the installer menu in 
the latest (06.09.2017) debian-9.0-sparc64-NETINST-1.iso image. Despite 
of that SILO is installed.


Regards,
Fedor


 Перенаправленное сообщение 
Тема:   Re: Latest Debian SPARC64 ISO Image
Дата:   Wed, 6 Sep 2017 12:05:09 +0200
От: John Paul Adrian Glaubitz 
Кому:   Fedor Konstantinov 



Can you please post to the mailing list, so that everyone can jump into 
the discussion?


Also, you don’t need to send screenshots.

Adrian

On Sep 6, 2017, at 11:47 AM, Fedor Konstantinov > wrote:



Adrian,

I found silo* packages are on their places. But it's still no "Install 
SILO boot loader" option in the installer menu.


Regards,
Fedor



06.09.2017 12:04, Fedor Konstantinov пишет:

Hi, Adrian.

Thank you for your help. As I can see you have already uploaded new 
image, so I'll try it.


Best regards,
Fedor

06.09.2017 01:57, John Paul Adrian Glaubitz пишет:

Hi!

On 09/06/2017 12:55 AM, James Clarke wrote:
Thanks for the report. This is a known bug[0]; some of the latest 
installer

images are missing needed packages from the unreleased suite.

I have already resolved the issue. I'm just waiting to get write
permission to the cdimage FTP server to upload the images [1].

Adrian


[1] https://cdimage.debian.org/cdimage/ports/

Re: Updated debian-installer images

2017-09-06 Thread James Clarke 
On 6 Sep 2017, at 11:27, Frank Scheiner  wrote:
> On 09/06/2017 09:37 AM, John Paul Adrian Glaubitz wrote:
>> [...]
>> FWIW, I have found the issue and the new images are ready. I'm currently
>> waiting to get write permissions to upload them here:
>>> https://cdimage.debian.org/cdimage/ports/
> 
> Thanks, I already grabbed a copy for testing.
> 
>> [...]
>> I have never tried to install Debian sparc64 on any non-Sun/Oracle machine so
>> I can't really tell you anything.
>>> [2]: https://wiki.debian.org/Sparc64#Installing_the_Debian_SPARC64_Port
>> We should probably ask the person who wrote this wiki page.
> 
> Ah I see, I assumed that would be you because of the info related to your 
> ISOs and the installation process on different machines. Is that James Clarke 
> then (just recognized [1])?
> 
> [1]: https://wiki.debian.org/Sparc64?action=info

I only made a couple of minor changes; the edit you want is [0] by Alex
McWhirter.

James

[0] https://wiki.debian.org/Sparc64?action=diff&rev1=25&rev2=26

Re: Updated debian-installer images

2017-09-06 Thread Frank Scheiner 

On 09/06/2017 09:37 AM, John Paul Adrian Glaubitz wrote:

[...]
FWIW, I have found the issue and the new images are ready. I'm currently
waiting to get write permissions to upload them here:


https://cdimage.debian.org/cdimage/ports/


Thanks, I already grabbed a copy for testing.


[...]

I have never tried to install Debian sparc64 on any non-Sun/Oracle machine so
I can't really tell you anything.


[2]: https://wiki.debian.org/Sparc64#Installing_the_Debian_SPARC64_Port


We should probably ask the person who wrote this wiki page.


Ah I see, I assumed that would be you because of the info related to 
your ISOs and the installation process on different machines. Is that 
James Clarke then (just recognized [1])?


[1]: https://wiki.debian.org/Sparc64?action=info

Re: Bad news

2017-09-06 Thread John Paul Adrian Glaubitz 
On 09/04/2017 05:29 PM, John Paul Adrian Glaubitz wrote:
> There is one talk about a SPARC emulator for x86, multiple talks about putting
> your stuff into Oracle's SPARC cloud, i.e. on actual SPARC hardware.
> 
> And companies trying to push you into the cloud is not Oracle-specific and 
> also
> not a sign of SPARC being killed. They just want you to come into the cloud
> because cloud services are basically a money-printing mechanism.
And now there is also this statement from Ala Coopersmith [1]:

"Unlike many friends, I’m still working at @Oracle on the next
 release of @OracleSolaris . Can't say yet how plans for it will change."

Adrian

> [1] https://twitter.com/alanc/status/904366563976896512

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

New download location for Debian Ports ISOs

2017-09-06 Thread John Paul Adrian Glaubitz 
Hi!

I am cross-posting this to debian-ports@l.d.o because it affects all
Debian Ports architectures.

As of today, we will be using Debian's official cdimage mirror to host
the installation images for Debian Ports, the images can be found in [1].

I have uploaded images for hppa, m68k, ppc64 and sparc64. I am working
on uploading more images. Images for alpha are currently missing due to
issues with the Linux kernel packages which fails to build on alpha at
the moment.

Please test the images as thoroughly as you can and report your results
to the appropriate architecture-specific mailing lists. Please do not
post to this mailing list, debian-ports@l.d.o, as this will cross-post
your mail to ALL Debian Ports architectures mailing lists.

A huge thanks to Steve McIntyre and the Debian Sysadmins for providing
us access to the official cdimage mirror.

Thanks,
Adrian

> [1] https://cdimage.debian.org/cdimage/ports/

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Updated debian-installer images

2017-09-06 Thread John Paul Adrian Glaubitz 
On 09/06/2017 08:40 AM, Frank Scheiner wrote:
>> For some reason the CD building process is omitting several packages
>> that should be included. I have not yet figured out what the problem
>> is. Still investigating.
> 
> Many thanks for your hard work on this.

FWIW, I have found the issue and the new images are ready. I'm currently
waiting to get write permissions to upload them here:

> https://cdimage.debian.org/cdimage/ports/

> In the meantime I tried to boot the earlier mentioned installer image also
> on a PRIMEPOWER 250 with SPARC64 V+ CPUs, just to check if the used Linux 
> kernel
> boots the machine - the missing SILO installer wouldn't be a problem during 
> this state.

Interesting.

> After the previous testing I came to the conclusion that Linux seems to not 
> support
> Fujitsu's SPARC64 CPUs (as per [1], "Note that Fujitsu's SPARC64 CPUs used
> in PRIMEPOWER family of servers are not supported due to lack of support in
> the Linux kernel. "), although it came pretty far for a "lack of support in 
> the
> Linux kernel". But as SPARC64 V and V+ are listed on [2] and the mentioned 
> info
> is from the Wheezy manual so maybe outdated, I again tried to boot with the
> latest image available.

I have never tried to install Debian sparc64 on any non-Sun/Oracle machine so
I can't really tell you anything.

> [2]: https://wiki.debian.org/Sparc64#Installing_the_Debian_SPARC64_Port

We should probably ask the person who wrote this wiki page.

> But sadly the machine also crashes early (log below), just like with the 
> previous installer image I tested. Should this actually work?

No idea. I would suggest taking this to the sparclinux LKML and
ask the kernel folks. They will know more. Maybe it's just a matter
of adjusting the kernel configuration. Maybe we need to create a different
kernel flavour for these machines, who knows?

Either way, this is something I can't do a alone.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913