Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Wed, 25 Aug 1999, Mark Newton wrote: Matthew Dillon wrote: The question I am putting to the group is whether it is "time" for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It makes sense; We wouldn't be the first to do it either (IRIX has supported 16 partitions per spindle for years). The same is true for OpenBSD. I too run out of partitions and am in favor of this change. I'm not sure, but I think the OpenBSD folks did something that allowed them to remain compatible with the 8 partition labels. In any case, it groks SunOS and Solaris labels on my OpenBSD sparc. Adrian -- [ [EMAIL PROTECTED] -- Ubergeeks Consulting -- http://www.ubergeeks.com/ ] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
What I would really like to do is to increase the number of partitions allowed in a disklabel. I really dislike having to mess with fdisk. The system defaults to 8. sys/diskslice.h seems to imply that you can compile up a kernel with a higher number. it looks you also have to change at sys/sys/disklabel.h and update the struct disklabel ? But you are right that the struct looks easy to change and we can go up to 22 partitions and remain within 512 bytes (seems there are 16 bytes/partition in struct disklabel, whereas struct diskslice seems not to pose problems. only thing to look at could be the boot blocks to be sure that they don't limit to read only the first 276 bytes (or if they do, you can't boot after the 8th partition). The structure appears to be backwards compatible. The question I am putting to the group is whether it is time for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p i guess one should try to increase to the max that fits a sector i.e 22 luigi To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Wed, 25 Aug 1999, Mark Newton wrote: Matthew Dillon wrote: The question I am putting to the group is whether it is time for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It makes sense; We wouldn't be the first to do it either (IRIX has supported 16 partitions per spindle for years). The same is true for OpenBSD. I too run out of partitions and am in favor of this change. I'm not sure, but I think the OpenBSD folks did something that allowed them to remain compatible with the 8 partition labels. In any case, it groks SunOS and Solaris labels on my OpenBSD sparc. Adrian -- [ adr...@ubergeeks.com -- Ubergeeks Consulting -- http://www.ubergeeks.com/ ] To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Possibility of increasing default MAXPARTITIONS from 8 to 16
I don't know about all of you, but for the last few years I've been running out of partitions! It's even worse with today's big disks. The last disk I installed I had to resort to using two fdisk slices on a single disk: apollo:/usr/src/sys# df Filesystem 1K-blocks UsedAvail Capacity Mounted on /dev/da0s1a127023528176404545%/ /dev/da0s1d127023 9035 107827 8%/var /dev/da0s1e127023 314 116548 0%/var/tmp /dev/da0s1f 1016303 435952 49904747%/usr /dev/da0s1g126322175459867215%/var/log /dev/da0s2a 1524463 631822 77068445%/archive /dev/da0s2d 3048942 1670121 113490660%/src /dev/da0s2e 5541549 1616954 348127232%/FreeBSD /dev/da0s2f 2032623 861297 100871746%/images /dev/da0s2g 3048942 243147 2561880 9%/usr/obj What I would really like to do is to increase the number of partitions allowed in a disklabel. I really dislike having to mess with fdisk. The system defaults to 8. sys/diskslice.h seems to imply that you can compile up a kernel with a higher number. The partition structure with 8 partitions eats 276 bytes. With 16 partitions it eats 404 bytes. I assume that anything under 512 bytes is safe, and possibly even more (how much does a UFS filesystem ignore?) The structure appears to be backwards compatible. The question I am putting to the group is whether it is "time" for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
What are the advantages and disadvantages of partitions? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
Matthew Dillon wrote: The question I am putting to the group is whether it is "time" for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It makes sense; We wouldn't be the first to do it either (IRIX has supported 16 partitions per spindle for years). Have you made the change on your hackbox already to make sure it doesn't have any negative implications? - mark Mark Newton Email: [EMAIL PROTECTED] (W) Network Engineer Email: [EMAIL PROTECTED] (H) Internode Systems Pty Ltd Desk: +61-8-82232999 "Network Man" - Anagram of "Mark Newton" Mobile: +61-416-202-223 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote: I don't know about all of you, but for the last few years I've been running out of partitions! It's even worse with today's big disks. I know it's not the answer, it's just related question: do you know perhaps of any initiatives (except XFS) that could significantly shorten time it takes fsck to check big filesystems, let's say 64GB? As it is now, it's almost unbearable. I naively thought softupdates would (almost) eliminate the need to do fsck... Andrzej Bialecki // [EMAIL PROTECTED] WebGiro AB, Sweden (http://www.webgiro.com) // --- // -- FreeBSD: The Power to Serve. http://www.freebsd.org // --- Small Embedded FreeBSD: http://www.freebsd.org/~picobsd/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
Andrzej Bialecki wrote: On Tue, 24 Aug 1999, Matthew Dillon wrote: I don't know about all of you, but for the last few years I've been running out of partitions! It's even worse with today's big disks. I know it's not the answer, it's just related question: do you know perhaps of any initiatives (except XFS) that could significantly shorten time it takes fsck to check big filesystems, let's say 64GB? As it is now, it's almost unbearable. I naively thought softupdates would (almost) eliminate the need to do fsck... The UFS checkpointing stuff Kirk is working on is supposed to be the magic bullet that fixes this. XFS will be kinda neat too. - mark Mark Newton Email: [EMAIL PROTECTED] (W) Network Engineer Email: [EMAIL PROTECTED] (H) Internode Systems Pty Ltd Desk: +61-8-82232999 "Network Man" - Anagram of "Mark Newton" Mobile: +61-416-202-223 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
The structure appears to be backwards compatible. The question I am putting to the group is whether it is "time" for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It seems reasonable to me, although there may be issues with finding a bit in the minor number - I think they've pretty much all been taken. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:It makes sense; We wouldn't be the first to do it either (IRIX has :supported 16 partitions per spindle for years). : :Have you made the change on your hackbox already to make sure it doesn't :have any negative implications? : :- mark : : :Mark Newton Email: [EMAIL PROTECTED] (W) If people seem generally for it I am willing to run a bunch of tests on my test boxes to ensure viability and backwards compatibility. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:I know it's not the answer, it's just related question: do you know :perhaps of any initiatives (except XFS) that could significantly shorten :time it takes fsck to check big filesystems, let's say 64GB? As it is now, :it's almost unbearable. I naively thought softupdates would (almost) :eliminate the need to do fsck... : :Andrzej Bialecki Eventually Kirk is planning for softupdates to allow you to run a special version of fsck in the background to clean up the block bitmap on a live filesystem. The time frame for this project is not known. Another possibility would be to mark individual cylinders clean/dirty to reduce the amount of work fsck must do on a normal filesystem. It would be a pretty hefty project for someone to take on, though. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:The question I am putting to the group is whether it is "time" for us, :with today's large disks, to increase the system-compiled default :from 8 to 16 partitions. Instead of a-h we would have a-p : : It seems reasonable to me, although there may be issues with finding a bit :in the minor number - I think they've pretty much all been taken. : :-DG : :David Greenman Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:: It seems reasonable to me, although there may be issues with finding a bit
::in the minor number - I think they've pretty much all been taken.
::
::-DG
::
::David Greenman
:
:Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8.
:
: -Matt
minor device number:
t = type
u = unit (upper bits split off from lower)
s = slice (assuming only 8 slices)
0 noslice
1 whole disk
2-5 s1,s2,s3,s4
6-8 unused
p = partition
- = cannot be used
tttu uuus uppp minor layout for disks
mask for uminor
Our only hope as far as I can tell is to take the upper two bits of the
slice (bits 19 and 20) and use them for the partition instead.
As far as I can tell, only slices 0-5 are used, requiring only 3 bits.
Can someone check my work? Are bits 19 and 20 unused by any disk
devices?
-Matt
Matthew Dillon
[EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
more... Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:minor device number: : :t = type : :u = unit (upper bits split off from lower) : :s = slice (assuming only 8 slices) : 0 noslice : 1 whole disk : 2-5 s1,s2,s3,s4 : 6-8 unused : :p = partition : :- = cannot be used : : : tttu uuus uppp minor layout for disks : mask for uminor : :Our only hope as far as I can tell is to take the upper two bits of the :slice (bits 19 and 20) and use them for the partition instead. : :As far as I can tell, only slices 0-5 are used, requiring only 3 bits. : :Can someone check my work? Are bits 19 and 20 unused by any disk :devices? : : -Matt Grrr. #ifdef PC98, NDOSPART is set to 16 instead of 4, requiring 5 bits (0,1,2-17). This leaves one bit that we can usurp. I'd rather have 2 bits, but I'll take 1 if I can get it. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, David Greenman wrote: The structure appears to be backwards compatible. The question I am putting to the group is whether it is "time" for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It seems reasonable to me, although there may be issues with finding a bit in the minor number - I think they've pretty much all been taken. Mph Mph! Mmmmph! (damned gags) -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote:
mask for uminor
Our only hope as far as I can tell is to take the upper two bits of the
slice (bits 19 and 20) and use them for the partition instead.
As far as I can tell, only slices 0-5 are used, requiring only 3 bits.
Can someone check my work? Are bits 19 and 20 unused by any disk
devices?
no there can be up to 32 slices (including the 2 special ones)
requiring 5 bits.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
According to Julian Elischer: Mph Mph! Mmmmph! (damned gags) devfs anyone ? [runs for cover] -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- [EMAIL PROTECTED] FreeBSD keltia.freenix.fr 4.0-CURRENT #73: Sat Jul 31 15:36:05 CEST 1999 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: more... Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:: tttu uuus uppp minor layout for disks :: mask for uminor :: :Grrr. #ifdef PC98, NDOSPART is set to 16 instead of 4, requiring 5 bits :(0,1,2-17). This leaves one bit that we can usurp. : :I'd rather have 2 bits, but I'll take 1 if I can get it. oops. only 5 bits of 's' anyway. A type bit ? 7 type bits, appear to be mostly unused for disks. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote: :I know it's not the answer, it's just related question: do you know :perhaps of any initiatives (except XFS) that could significantly shorten :time it takes fsck to check big filesystems, let's say 64GB? As it is now, :it's almost unbearable. I naively thought softupdates would (almost) :eliminate the need to do fsck... : :Andrzej Bialecki Eventually Kirk is planning for softupdates to allow you to run a special version of fsck in the background to clean up the block bitmap on a live filesystem. The time frame for this project is not known. Another possibility would be to mark individual cylinders clean/dirty to reduce the amount of work fsck must do on a normal filesystem. It would be a pretty hefty project for someone to take on, though. Hmm.. If I understand you correctly: * the ffs code would have to be modified to mark cylinder groups "dirty" when there are writes to that CG. * on unmount, after the buffers are flushed they would be marked clean. * on mount all "clean" flags in CGs would have to be ckecked (instead of the single bit) * fsck would have to be modified to recognize CG "clean" flag and prune only those CGs. Overall, doesn't sound _that_ complicated... but most probably I'm missing something. Andrzej Bialecki // [EMAIL PROTECTED] WebGiro AB, Sweden (http://www.webgiro.com) // --- // -- FreeBSD: The Power to Serve. http://www.freebsd.org // --- Small Embedded FreeBSD: http://www.freebsd.org/~picobsd/ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:The question I am putting to the group is whether it is "time" for us, :with today's large disks, to increase the system-compiled default :from 8 to 16 partitions. Instead of a-h we would have a-p : : It seems reasonable to me, although there may be issues with finding a bit :in the minor number - I think they've pretty much all been taken. Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. Use devfs (it doesn't require major/minor numbers). Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
: Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. : :Use devfs (it doesn't require major/minor numbers). : : Terry Lambert : [EMAIL PROTECTED] We can't use devfs for this, the partition id is already embedded in the dev_t and it has to stay that way to remain compatible with the preexisting partition space. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote: : Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. : :Use devfs (it doesn't require major/minor numbers). : : Terry Lambert : [EMAIL PROTECTED] We can't use devfs for this, the partition id is already embedded in the dev_t and it has to stay that way to remain compatible with the preexisting partition space. A true devfs implementation (such as was deleted by Vikings inc.) can allocate the minor numbers arbitrarily. They are simply cookies and have no embedded meaning.. SLICE started with 0 for the first disk-like object found and incemented each time it found another. I'm very close to having it all working again.. -Matt Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:I know it's not the answer, it's just related question: do you know :perhaps of any initiatives (except XFS) that could significantly shorten :time it takes fsck to check big filesystems, let's say 64GB? As it is now, :it's almost unbearable. I naively thought softupdates would (almost) :eliminate the need to do fsck... : :Andrzej Bialecki Eventually Kirk is planning for softupdates to allow you to run a special version of fsck in the background to clean up the block bitmap on a live filesystem. The time frame for this project is not known. Another possibility would be to mark individual cylinders clean/dirty to reduce the amount of work fsck must do on a normal filesystem. It would be a pretty hefty project for someone to take on, though. Hmm.. If I understand you correctly: * the ffs code would have to be modified to mark cylinder groups "dirty" when there are writes to that CG. * on unmount, after the buffers are flushed they would be marked clean. * on mount all "clean" flags in CGs would have to be ckecked (instead of the single bit) * fsck would have to be modified to recognize CG "clean" flag and prune only those CGs. Overall, doesn't sound _that_ complicated... but most probably I'm missing something. When a system crashes, the dirty bit is set. Because the dirty bit is set, you can't trust the FS contents to be able to distinguish between a crash that was the result of a software failure, and one that was the result of a hardware failure. Because of this, you must assume a hardware failure, and engage in a full check. In the case of a software failure, the cylinder group bitmaps may, in fact, have bits indicating that things which are not truly allocated have in fact been allocated. The process of traversing these (locking each CG as you do so) to clear the bits on things that were never truly allocated is the "fsck in the background" operation which is permissible following a software failure which leaves the dirty bit set for the FS. There are two rational methods for getting around this problem; the first was suggested by Ganger and Patt, Matt Day, Mark Muhlestein, myself, and others: "soft read-only". A "soft read-only" implementation was done (by Kirk) for BSDI. The basic idea is to mark the in core superblock read-only after there are no dirty buffer left associated with an FS, and then mark the on-disk structure clean. When a write (or a read, since you must obey POSIX atime semantics) occurs, you must mark the FS dirty and _be certain this write has been commited to disk_, before clearing the "soft read-only" flag and allowing the dirtying operation to complete. An implementation of this is pretty trivial on a normal system, and Matt, Mark, and myself implemented such a beast for our Windows 95 port of the Heidemann framework and the BSD FFS (and the Ganger/Patt Soft updates code). This gives you a sort of "statistical protection", which is most useful for a single user desktop box (e.g. Windows 95), where the box's disks are frequently idle for large stretches of time, and therefore in the state "clean, soft-read-only". For FreeBSD, the problem is complicated by the FS metadata's dirty buffers being hung off the device vnode, rather than being truly seperate data. This means that you must sync out that data, as well, before you can mark the FS clean (and you must resync out similar data to besure the dirty bit has been correctly set, before proceeding with other writes). For the Windows 95 port of the code, there was no unified VM and buffer cache to have to worry about in this regard. Apart from "soft read-only", you can obtain, at the cylinder group level, seperate "clean bits" on a per cylinder group basis. For this to work in the face of a true hardware failure, you must engage in a two stage commit process, in which you mark the entire FS dirty, modify the state of the cylinder group clean bit, and then mark the FS clean. This works in the face of software failures for cylinder group operations. To make it robust in the face of hardware failures, you must have a seperate "dirty-but-ok" bit for the cylinder group, which is similarly protected, and which is reset (after resetting the FS dirty bit, after resetting the CG dirty bit) during updates to non-CG bitmap data. Failure to support this leaves you unable to verify the state of the non-bitmap data in the CG bitmap, particularly for files whose block pointers span cylinder groups. Processing of cleanup is further complicated by the fact that any file that spans a "dirty, dirty" CG after a software failure must be treated as if it had been involved in a hardware failure. With a large number of files, the benefits gained by this approach are small. Aside: I was under the impresssion from the Usenix reports that Kirk's checkpointing mechanism was a reference to the ability to stall an image of the FS as an exposed "snapshot", to allow for backups to
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
: Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. : :Use devfs (it doesn't require major/minor numbers). We can't use devfs for this, the partition id is already embedded in the dev_t and it has to stay that way to remain compatible with the preexisting partition space. Too bad you are not as willing to abandon your existing partition space as quickly as you are willing to abandon your historical disklabel format, or you wouldn't be having this problem. 8-). Terry Lambert [EMAIL PROTECTED] --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
: :Use devfs (it doesn't require major/minor numbers). : : We can't use devfs for this, the partition id is already embedded in the : dev_t and it has to stay that way to remain compatible with the : preexisting partition space. : :Too bad you are not as willing to abandon your existing partition :space as quickly as you are willing to abandon your historical :disklabel format, or you wouldn't be having this problem. 8-). : : Terry Lambert : [EMAIL PROTECTED] When devfs is the only way to access devices we can abandon traditional device major/minor id's. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:The question I am putting to the group is whether it is "time" for us, :with today's large disks, to increase the system-compiled default :from 8 to 16 partitions. Instead of a-h we would have a-p : : It seems reasonable to me, although there may be issues with finding a bit :in the minor number - I think they've pretty much all been taken. : :-DG : :David Greenman Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. DEVFS/SLICE to the rescue. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ [EMAIL PROTECTED] \\-- Joseph Merrick \\ [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
What I would really like to do is to increase the number of partitions allowed in a disklabel. I really dislike having to mess with fdisk. The system defaults to 8. sys/diskslice.h seems to imply that you can compile up a kernel with a higher number. it looks you also have to change at sys/sys/disklabel.h and update the struct disklabel ? But you are right that the struct looks easy to change and we can go up to 22 partitions and remain within 512 bytes (seems there are 16 bytes/partition in struct disklabel, whereas struct diskslice seems not to pose problems. only thing to look at could be the boot blocks to be sure that they don't limit to read only the first 276 bytes (or if they do, you can't boot after the 8th partition). The structure appears to be backwards compatible. The question I am putting to the group is whether it is "time" for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p i guess one should try to increase to the max that fits a sector i.e 22 luigi To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Possibility of increasing default MAXPARTITIONS from 8 to 16
I don't know about all of you, but for the last few years I've been running out of partitions! It's even worse with today's big disks. The last disk I installed I had to resort to using two fdisk slices on a single disk: apollo:/usr/src/sys# df Filesystem 1K-blocks UsedAvail Capacity Mounted on /dev/da0s1a127023528176404545%/ /dev/da0s1d127023 9035 107827 8%/var /dev/da0s1e127023 314 116548 0%/var/tmp /dev/da0s1f 1016303 435952 49904747%/usr /dev/da0s1g126322175459867215%/var/log /dev/da0s2a 1524463 631822 77068445%/archive /dev/da0s2d 3048942 1670121 113490660%/src /dev/da0s2e 5541549 1616954 348127232%/FreeBSD /dev/da0s2f 2032623 861297 100871746%/images /dev/da0s2g 3048942 243147 2561880 9%/usr/obj What I would really like to do is to increase the number of partitions allowed in a disklabel. I really dislike having to mess with fdisk. The system defaults to 8. sys/diskslice.h seems to imply that you can compile up a kernel with a higher number. The partition structure with 8 partitions eats 276 bytes. With 16 partitions it eats 404 bytes. I assume that anything under 512 bytes is safe, and possibly even more (how much does a UFS filesystem ignore?) The structure appears to be backwards compatible. The question I am putting to the group is whether it is time for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p -Matt To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
What are the advantages and disadvantages of partitions? To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
Matthew Dillon wrote: The question I am putting to the group is whether it is time for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It makes sense; We wouldn't be the first to do it either (IRIX has supported 16 partitions per spindle for years). Have you made the change on your hackbox already to make sure it doesn't have any negative implications? - mark Mark Newton Email: new...@internode.com.au (W) Network Engineer Email: new...@atdot.dotat.org (H) Internode Systems Pty Ltd Desk: +61-8-82232999 Network Man - Anagram of Mark Newton Mobile: +61-416-202-223 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote: I don't know about all of you, but for the last few years I've been running out of partitions! It's even worse with today's big disks. I know it's not the answer, it's just related question: do you know perhaps of any initiatives (except XFS) that could significantly shorten time it takes fsck to check big filesystems, let's say 64GB? As it is now, it's almost unbearable. I naively thought softupdates would (almost) eliminate the need to do fsck... Andrzej Bialecki // ab...@webgiro.com WebGiro AB, Sweden (http://www.webgiro.com) // --- // -- FreeBSD: The Power to Serve. http://www.freebsd.org // --- Small Embedded FreeBSD: http://www.freebsd.org/~picobsd/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
Andrzej Bialecki wrote: On Tue, 24 Aug 1999, Matthew Dillon wrote: I don't know about all of you, but for the last few years I've been running out of partitions! It's even worse with today's big disks. I know it's not the answer, it's just related question: do you know perhaps of any initiatives (except XFS) that could significantly shorten time it takes fsck to check big filesystems, let's say 64GB? As it is now, it's almost unbearable. I naively thought softupdates would (almost) eliminate the need to do fsck... The UFS checkpointing stuff Kirk is working on is supposed to be the magic bullet that fixes this. XFS will be kinda neat too. - mark Mark Newton Email: new...@internode.com.au (W) Network Engineer Email: new...@atdot.dotat.org (H) Internode Systems Pty Ltd Desk: +61-8-82232999 Network Man - Anagram of Mark Newton Mobile: +61-416-202-223 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
The structure appears to be backwards compatible. The question I am putting to the group is whether it is time for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It seems reasonable to me, although there may be issues with finding a bit in the minor number - I think they've pretty much all been taken. -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:It makes sense; We wouldn't be the first to do it either (IRIX has :supported 16 partitions per spindle for years). : :Have you made the change on your hackbox already to make sure it doesn't :have any negative implications? : :- mark : : :Mark Newton Email: new...@internode.com.au (W) If people seem generally for it I am willing to run a bunch of tests on my test boxes to ensure viability and backwards compatibility. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:I know it's not the answer, it's just related question: do you know :perhaps of any initiatives (except XFS) that could significantly shorten :time it takes fsck to check big filesystems, let's say 64GB? As it is now, :it's almost unbearable. I naively thought softupdates would (almost) :eliminate the need to do fsck... : :Andrzej Bialecki Eventually Kirk is planning for softupdates to allow you to run a special version of fsck in the background to clean up the block bitmap on a live filesystem. The time frame for this project is not known. Another possibility would be to mark individual cylinders clean/dirty to reduce the amount of work fsck must do on a normal filesystem. It would be a pretty hefty project for someone to take on, though. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:The question I am putting to the group is whether it is time for us, :with today's large disks, to increase the system-compiled default :from 8 to 16 partitions. Instead of a-h we would have a-p : : It seems reasonable to me, although there may be issues with finding a bit :in the minor number - I think they've pretty much all been taken. : :-DG : :David Greenman Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:: It seems reasonable to me, although there may be issues with finding a bit
::in the minor number - I think they've pretty much all been taken.
::
::-DG
::
::David Greenman
:
:Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8.
:
: -Matt
minor device number:
t = type
u = unit (upper bits split off from lower)
s = slice (assuming only 8 slices)
0 noslice
1 whole disk
2-5 s1,s2,s3,s4
6-8 unused
p = partition
- = cannot be used
tttu uuus uppp minor layout for disks
mask for uminor
Our only hope as far as I can tell is to take the upper two bits of the
slice (bits 19 and 20) and use them for the partition instead.
As far as I can tell, only slices 0-5 are used, requiring only 3 bits.
Can someone check my work? Are bits 19 and 20 unused by any disk
devices?
-Matt
Matthew Dillon
dil...@backplane.com
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
more... Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:minor device number: : :t = type : :u = unit (upper bits split off from lower) : :s = slice (assuming only 8 slices) : 0 noslice : 1 whole disk : 2-5 s1,s2,s3,s4 : 6-8 unused : :p = partition : :- = cannot be used : : : tttu uuus uppp minor layout for disks : mask for uminor : :Our only hope as far as I can tell is to take the upper two bits of the :slice (bits 19 and 20) and use them for the partition instead. : :As far as I can tell, only slices 0-5 are used, requiring only 3 bits. : :Can someone check my work? Are bits 19 and 20 unused by any disk :devices? : : -Matt Grrr. #ifdef PC98, NDOSPART is set to 16 instead of 4, requiring 5 bits (0,1,2-17). This leaves one bit that we can usurp. I'd rather have 2 bits, but I'll take 1 if I can get it. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, David Greenman wrote: The structure appears to be backwards compatible. The question I am putting to the group is whether it is time for us, with today's large disks, to increase the system-compiled default from 8 to 16 partitions. Instead of a-h we would have a-p It seems reasonable to me, although there may be issues with finding a bit in the minor number - I think they've pretty much all been taken. Mph Mph! Mmmmph! (damned gags) -DG David Greenman Co-founder/Principal Architect, The FreeBSD Project - http://www.freebsd.org Creator of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote:
mask for uminor
Our only hope as far as I can tell is to take the upper two bits of the
slice (bits 19 and 20) and use them for the partition instead.
As far as I can tell, only slices 0-5 are used, requiring only 3 bits.
Can someone check my work? Are bits 19 and 20 unused by any disk
devices?
no there can be up to 32 slices (including the 2 special ones)
requiring 5 bits.
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
According to Julian Elischer: Mph Mph! Mmmmph! (damned gags) devfs anyone ? [runs for cover] -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- robe...@keltia.freenix.fr FreeBSD keltia.freenix.fr 4.0-CURRENT #73: Sat Jul 31 15:36:05 CEST 1999 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: more... Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:: tttu uuus uppp minor layout for disks :: mask for uminor :: :Grrr. #ifdef PC98, NDOSPART is set to 16 instead of 4, requiring 5 bits :(0,1,2-17). This leaves one bit that we can usurp. : :I'd rather have 2 bits, but I'll take 1 if I can get it. oops. only 5 bits of 's' anyway. A type bit ? 7 type bits, appear to be mostly unused for disks. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote: :I know it's not the answer, it's just related question: do you know :perhaps of any initiatives (except XFS) that could significantly shorten :time it takes fsck to check big filesystems, let's say 64GB? As it is now, :it's almost unbearable. I naively thought softupdates would (almost) :eliminate the need to do fsck... : :Andrzej Bialecki Eventually Kirk is planning for softupdates to allow you to run a special version of fsck in the background to clean up the block bitmap on a live filesystem. The time frame for this project is not known. Another possibility would be to mark individual cylinders clean/dirty to reduce the amount of work fsck must do on a normal filesystem. It would be a pretty hefty project for someone to take on, though. Hmm.. If I understand you correctly: * the ffs code would have to be modified to mark cylinder groups dirty when there are writes to that CG. * on unmount, after the buffers are flushed they would be marked clean. * on mount all clean flags in CGs would have to be ckecked (instead of the single bit) * fsck would have to be modified to recognize CG clean flag and prune only those CGs. Overall, doesn't sound _that_ complicated... but most probably I'm missing something. Andrzej Bialecki // ab...@webgiro.com WebGiro AB, Sweden (http://www.webgiro.com) // --- // -- FreeBSD: The Power to Serve. http://www.freebsd.org // --- Small Embedded FreeBSD: http://www.freebsd.org/~picobsd/ To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:The question I am putting to the group is whether it is time for us, :with today's large disks, to increase the system-compiled default :from 8 to 16 partitions. Instead of a-h we would have a-p : : It seems reasonable to me, although there may be issues with finding a bit :in the minor number - I think they've pretty much all been taken. Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. Use devfs (it doesn't require major/minor numbers). Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
: Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. : :Use devfs (it doesn't require major/minor numbers). : : Terry Lambert : te...@lambert.org We can't use devfs for this, the partition id is already embedded in the dev_t and it has to stay that way to remain compatible with the preexisting partition space. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
On Tue, 24 Aug 1999, Matthew Dillon wrote: : Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. : :Use devfs (it doesn't require major/minor numbers). : : Terry Lambert : te...@lambert.org We can't use devfs for this, the partition id is already embedded in the dev_t and it has to stay that way to remain compatible with the preexisting partition space. A true devfs implementation (such as was deleted by Vikings inc.) can allocate the minor numbers arbitrarily. They are simply cookies and have no embedded meaning.. SLICE started with 0 for the first disk-like object found and incemented each time it found another. I'm very close to having it all working again.. -Matt Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:I know it's not the answer, it's just related question: do you know :perhaps of any initiatives (except XFS) that could significantly shorten :time it takes fsck to check big filesystems, let's say 64GB? As it is now, :it's almost unbearable. I naively thought softupdates would (almost) :eliminate the need to do fsck... : :Andrzej Bialecki Eventually Kirk is planning for softupdates to allow you to run a special version of fsck in the background to clean up the block bitmap on a live filesystem. The time frame for this project is not known. Another possibility would be to mark individual cylinders clean/dirty to reduce the amount of work fsck must do on a normal filesystem. It would be a pretty hefty project for someone to take on, though. Hmm.. If I understand you correctly: * the ffs code would have to be modified to mark cylinder groups dirty when there are writes to that CG. * on unmount, after the buffers are flushed they would be marked clean. * on mount all clean flags in CGs would have to be ckecked (instead of the single bit) * fsck would have to be modified to recognize CG clean flag and prune only those CGs. Overall, doesn't sound _that_ complicated... but most probably I'm missing something. When a system crashes, the dirty bit is set. Because the dirty bit is set, you can't trust the FS contents to be able to distinguish between a crash that was the result of a software failure, and one that was the result of a hardware failure. Because of this, you must assume a hardware failure, and engage in a full check. In the case of a software failure, the cylinder group bitmaps may, in fact, have bits indicating that things which are not truly allocated have in fact been allocated. The process of traversing these (locking each CG as you do so) to clear the bits on things that were never truly allocated is the fsck in the background operation which is permissible following a software failure which leaves the dirty bit set for the FS. There are two rational methods for getting around this problem; the first was suggested by Ganger and Patt, Matt Day, Mark Muhlestein, myself, and others: soft read-only. A soft read-only implementation was done (by Kirk) for BSDI. The basic idea is to mark the in core superblock read-only after there are no dirty buffer left associated with an FS, and then mark the on-disk structure clean. When a write (or a read, since you must obey POSIX atime semantics) occurs, you must mark the FS dirty and _be certain this write has been commited to disk_, before clearing the soft read-only flag and allowing the dirtying operation to complete. An implementation of this is pretty trivial on a normal system, and Matt, Mark, and myself implemented such a beast for our Windows 95 port of the Heidemann framework and the BSD FFS (and the Ganger/Patt Soft updates code). This gives you a sort of statistical protection, which is most useful for a single user desktop box (e.g. Windows 95), where the box's disks are frequently idle for large stretches of time, and therefore in the state clean, soft-read-only. For FreeBSD, the problem is complicated by the FS metadata's dirty buffers being hung off the device vnode, rather than being truly seperate data. This means that you must sync out that data, as well, before you can mark the FS clean (and you must resync out similar data to besure the dirty bit has been correctly set, before proceeding with other writes). For the Windows 95 port of the code, there was no unified VM and buffer cache to have to worry about in this regard. Apart from soft read-only, you can obtain, at the cylinder group level, seperate clean bits on a per cylinder group basis. For this to work in the face of a true hardware failure, you must engage in a two stage commit process, in which you mark the entire FS dirty, modify the state of the cylinder group clean bit, and then mark the FS clean. This works in the face of software failures for cylinder group operations. To make it robust in the face of hardware failures, you must have a seperate dirty-but-ok bit for the cylinder group, which is similarly protected, and which is reset (after resetting the FS dirty bit, after resetting the CG dirty bit) during updates to non-CG bitmap data. Failure to support this leaves you unable to verify the state of the non-bitmap data in the CG bitmap, particularly for files whose block pointers span cylinder groups. Processing of cleanup is further complicated by the fact that any file that spans a dirty, dirty CG after a software failure must be treated as if it had been involved in a hardware failure. With a large number of files, the benefits gained by this approach are small. Aside: I was under the impresssion from the Usenix reports that Kirk's checkpointing mechanism was a reference to the ability to stall an image of the FS as an exposed snapshot, to allow for backups to occur on running FS's (and
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
: Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. : :Use devfs (it doesn't require major/minor numbers). We can't use devfs for this, the partition id is already embedded in the dev_t and it has to stay that way to remain compatible with the preexisting partition space. Too bad you are not as willing to abandon your existing partition space as quickly as you are willing to abandon your historical disklabel format, or you wouldn't be having this problem. 8-). Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
: :Use devfs (it doesn't require major/minor numbers). : : We can't use devfs for this, the partition id is already embedded in the : dev_t and it has to stay that way to remain compatible with the : preexisting partition space. : :Too bad you are not as willing to abandon your existing partition :space as quickly as you are willing to abandon your historical :disklabel format, or you wouldn't be having this problem. 8-). : : Terry Lambert : te...@lambert.org When devfs is the only way to access devices we can abandon traditional device major/minor id's. -Matt To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Possibility of increasing default MAXPARTITIONS from 8 to 16
:The question I am putting to the group is whether it is time for us, :with today's large disks, to increase the system-compiled default :from 8 to 16 partitions. Instead of a-h we would have a-p : : It seems reasonable to me, although there may be issues with finding a bit :in the minor number - I think they've pretty much all been taken. : :-DG : :David Greenman Cripes! Now that is really annoying. da0a is 4,0 and da1a is 4,8. DEVFS/SLICE to the rescue. -- \\ The mind's the standard \\ Mike Smith \\ of the man. \\ msm...@freebsd.org \\-- Joseph Merrick \\ msm...@cdrom.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
