Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread Rhialto
On Thu 03 Oct 2019 at 10:17:47 -0700, John Nemeth wrote:
> On Oct 3,  2:42pm, Rhialto wrote:
> } I was thinking the other day that it might be useful if gpt had a
> } subcommand to spit out a script to duplicate the partitioning of a disk,
> } but without the "unique" parts. The script would of course be
> } hand-editable for any changes one might want to make.
> 
>  By "unique" parts, do you mean just the UUIDs, or do you mean
> other parts as well?  What would the output look like?

Just the UUIDs, those are the things I'm aware of.

The sort of script I was thinking of would be something like this:

gpt destroy sd1
gpt create sd1
gpt add -a 2m -l efi   -s  128m -t efi  sd1
gpt add -a 2m -l root  -s  768m -t ffs  sd1
gpt add -a 2m -l var   -s3g -t ffs  sd1
gpt add -a 2m -l swap  -s 8200m -t swap sd1
gpt add -a 2m -l usr   -s   20g -t ffs  sd1
gpt add -a 2m -l tmp   -s 1024m -t ffs  sd1
gpt add -a 2m -l home  -s  128g -t ffs  sd1

It would leave out all/most things that are defaulted.
(probably when automatically generated it would use the -a alignment
only on the first partition, or if there is unused space between them)
(maybe it would even use some shell variables for things like the disk
device)

>  The bit about not modifying it is more of a caution.  You can
> modify it if you're careful (checksums are recomputed during
> restore).  Obviously if you make an error, it may not be usable
> for restore, so you should work with a copy.

Yes, when I wanted to clone the partitioning of my disk, I was indeed
thinking of modifying the output from 'gpt backup', but it seemed a bit
easier to create a little script such as the above, in case I want to
make the same partitions again later. (In fact the version that I really
used had some "are you sure" type prompts in it)

-Olaf.
-- 
Olaf 'Rhialto' Seibert -- rhialto at falu dot nl
___  Anyone who is capable of getting themselves made President should on
\X/  no account be allowed to do the job.   --Douglas Adams, "THGTTG"


signature.asc
Description: PGP signature


Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread John Nemeth
On Oct 3,  2:42pm, Rhialto wrote:
} On Wed 02 Oct 2019 at 19:40:01 -0700, John Nemeth wrote:
} >  Cloning disks always presents issues.  However, gpt(8) has
} > grown a "uuid" command to generate new UUIDs.  This was primarily
} > done to help with the cloning problem.  Cloning a disk and then
} > putting two disks with the same UUIDs on the same system is an
} > operator error.
} 
} I was thinking the other day that it might be useful if gpt had a
} subcommand to spit out a script to duplicate the partitioning of a disk,
} but without the "unique" parts. The script would of course be
} hand-editable for any changes one might want to make.

 By "unique" parts, do you mean just the UUIDs, or do you mean
other parts as well?  What would the output look like?

} Such functionality would be the equivalent of using disklabel to get the
} editable text-version of a disklabel from one disk, and applying it to a
} different disk.
} 
} I use this sort of thing if I want to create a backup disk which should
} have the same layout as the original.
} 
} There is "gpt backup", but the manual page mentions "It should not be
} modified" and it contains the guids of the partition which in many cases
} should NOT be duplicated. It also isn't very readable.

 The bit about not modifying it is more of a caution.  You can
modify it if you're careful (checksums are recomputed during
restore).  Obviously if you make an error, it may not be usable
for restore, so you should work with a copy.

}-- End of excerpt from Rhialto


Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread Martin Husemann
On Thu, Oct 03, 2019 at 09:51:40AM -0600, Warner Losh wrote:
> NetBSD is, of course, free to do what it likes. My semi-outsider's view
> suggests, though, that the FreeBSD experience is relevant and timely.

So this discussion wandered off topic a bit. Of course NetBSD already
supports gpt and there is no plan to change (legacy) disklabels.

Actually the whole original issue was triggered by my changes to the
installer being too strict with disklabels on legacy devices on certain
machines, where it now would reject a native disk and instead ask for
the prefered partitioning scheme - which is not a good idea when performing
an update.

I have a good solution for that in userland now w/o any kernel support 
required.

Martin


Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread Warner Losh
On Thu, Oct 3, 2019 at 9:19 AM Robert Elz  wrote:

>  Now it makes no sense at all.
>

FreeBSD made the explicit decision when disks were sneaking up on 2TB to
move to GPT labels. Why invent a new scheme that interoperates poorly with
other things? GPT, for better or worse, won. disklabel64 would add no value
over GPT, require a lot of extra code and be an ongoing source of confusion
and difficulty for our users. This is why UFS2 didn't bring in a 64-bit
disklabel format...

NetBSD is, of course, free to do what it likes. My semi-outsider's view
suggests, though, that the FreeBSD experience is relevant and timely.

Warner


Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread Robert Elz
Date:Wed, 2 Oct 2019 21:47:38 -0400 (EDT)
From:Mouse 
Message-ID:  <201910030147.vaa03...@stone.rodents-montreal.org>

John Nemeth said pretty much exactly what I would have said, but
there are a few points on which I would like to expand...

  | - Partitions are huge.  128 bytes,

Who cares?   These days I (and may others) align partitions on
fairly large boundaries (I use 1MB) for (some) future proofing of drive
technology - that means that the space between 0 and 1MB is unused
for any filesystem purpose, we may as well use all of it for labels
(though even the biggest GPT label doesn't come near to that).

Back in the 1970's when labels (MBR and disklabel) were invented,
trying to compress them into the smallest possible space made sense.
Now it is simply false ecomomy, saving a few meaningless sectors
is irrelevant, losing flexibility is a killer.

  | - Partition types are UUIDs.  Why use 16 bytes when you have, in the
  |information-theoretic sense, maybe three bits of information?

We know it isn't 3, the 8 bits that MBRs allowed is almost all used up.
You might only need a few, but other people need a different few (you
omitted LFS, ZFS, HFS, ...)

But what really matters is that any small number scheme needs a central
registry, and someone to maintain it - and then some mechanism to extend
it when it starts to run out.

Any new partition scheme designed now really needs to be planned to
last for at least as long, hopefully 10 times as long, as disklabels
and MBR have lasted (40 years) - with who knows what kinds of uses 
of it into the future.

Anyone ought to be able to design a new filesystem and give it a type
which won't conflict with others (MBR types already failed at that,
disklabels are used in such limited systems that it hasn't been such
a problem).   And they ought to be able to do that without having to
deal with a central bureaucracy which might say "no" or "use one of the
private use types we have defined until there are millions of users,
then we'll allocate a real number, and all those millions can be changed",
and "by the way, where do we send the invoice for this service?"

  | - Partition names(!) are character strings, rather than octet strings

The thing that matters is actually a UUID, what you're referring to is
the one thing that is missing from MBR and disklabels that I miss most.

I can see from the label that there are N partitions on the drive,
and what generic type they are (7 FFS and 1 swap) - but which one is
which?   Currently we do that (for FFS anyway) by checking the "last
mounted" in the superblock, but that is unreliable (doesn't work at all
for read only mounted filesystems which might never have been mounted
RW, and if they were might have been on /mnt when they were) and isn't
available at all in many other filesystem types (which of these swap
partitions had dumps enabled on it?)

A human readable name, in the label (ie: associated with the partition,
not the filesystem) can help with that it - it even assists during planning
when no filesystems have yet been created, just to make it easier to keep
track of which have had space allocated.   They're not required (by anything)
to be unique, or really usable for much, but they're an incredibly useful
thing to have.

  |It most certainly is
  |not appropriate for the partition table format to mandate/demand the
  |presence of Unicode support in the operating system.

Nothing of the kind is required, the OS doesn't need to use the label (name)
at all if it doesn't want (for a long time, NetBSD used GPT without doing
so in the kernel - only recently has some of this been added.)

  | It does have one nice thing, a (supposedly-)unique partition
  | identifier.

That's the real name (identifier) of the partition, not the name.

  | (Of course, nothing can ensure this actually _is_ unique;

No, but statistically, cloning aside (where sometimes being not unique
is exactly what is required, a clone should be identical) they are
very likely unique, and if generater properly, I don't imagine I'll
even see a (non-clone) duplicate.   I don't imagine you will either,
unless you force it.

  | No, not to be different.  To be better.
  |
  | Except, of course, that (I assume) you don't think it _would_ be
  | better.

It is entirely possible that one might design a partitioning scheme
that is better than GPT.   But by your argument I couldn't use that
one, as it is just as possible to design one even better still, and
even that one isn't usable for the same reason.

At some point we (the world, not NetBSD) needs to simply say "good enough"
and stop arguing.   GPT is certainly good enough.   Much better than
disklabels or MBR (aside from anything else it has no fixed maximum number
of partitions ... a particular label can only handle some number, but you
get to decide when you create the label how many you need).   Further
everything you don't like about it, I consider a step in the

Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread Martin Husemann
On Thu, Oct 03, 2019 at 02:42:10PM +0200, Rhialto wrote:
> I was thinking the other day that it might be useful if gpt had a
> subcommand to spit out a script to duplicate the partitioning of a disk,
> but without the "unique" parts. The script would of course be
> hand-editable for any changes one might want to make.

Just create a backup of the gpt (gpt backup [-o outfile]), restore it
on the other disk (gpt restore [-F] [-i infile]) and swap the IDs
(gpt uuid [-a]).

In a few days you will also be able to use sysinst to do this (optionally
including the binary content of the partitions or just the partitions
so you can apply more efficient options for the content), this is a side
effect of some improvements I am currently working on.

Martin


Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread Rhialto
On Wed 02 Oct 2019 at 19:40:01 -0700, John Nemeth wrote:
>  Cloning disks always presents issues.  However, gpt(8) has
> grown a "uuid" command to generate new UUIDs.  This was primarily
> done to help with the cloning problem.  Cloning a disk and then
> putting two disks with the same UUIDs on the same system is an
> operator error.

I was thinking the other day that it might be useful if gpt had a
subcommand to spit out a script to duplicate the partitioning of a disk,
but without the "unique" parts. The script would of course be
hand-editable for any changes one might want to make.

Such functionality would be the equivalent of using disklabel to get the
editable text-version of a disklabel from one disk, and applying it to a
different disk.

I use this sort of thing if I want to create a backup disk which should
have the same layout as the original.

There is "gpt backup", but the manual page mentions "It should not be
modified" and it contains the guids of the partition which in many cases
should NOT be duplicated. It also isn't very readable.

The intermediate solution I have used so far is to partition my disks in
all cases using a simple script, and keep the script corresponding to
each disk, so that I can use it again later. But if you didn't
originally partition a disk this way, it doesn't help.

-Olaf.
-- 
Olaf 'Rhialto' Seibert -- rhialto at falu dot nl
___  Anyone who is capable of getting themselves made President should on
\X/  no account be allowed to do the job.   --Douglas Adams, "THGTTG"


signature.asc
Description: PGP signature


Re: Adding an ioctl to check for disklabel existence

2019-10-03 Thread David Brownlee
While I agree NetBSD needs to support and work well with GPT in order
to interoperate with other systems, there is also prior art in
extending disklabel to 64bits - OpenBSD did this back in 2007 (though
there were a fair few follow up commits to cleanup the fallout :)

https://github.com/openbsd/src/commit/ddfcbf38c8ab6225a6b172d829aa957007d2587f#diff-192d23728acf9d8a70ab7259784d4162

David