On Mon, Dec 10, 2018 at 11:44:47AM +0100, Otto Moerbeek wrote:
> On Mon, Dec 10, 2018 at 08:30:10AM +0100, Otto Moerbeek wrote:
>
> > Hi,
> >
> > the bootloader uses a very simple allocator for dynamic memory. It
> > maintains a list of free allocations. If it needs a block, it searches
> > the freelist and returns the smallest allocation that fits.
> >
> > Allocation patterns like this (starting with an empty freelist)
> >
> > alloc(big)
> > free(big)
> > alloc(small)
> >
> > will assigned a big block for the small allocation, wasting most
> > memory. The allocator does not split up this block. After this, a new
> > big allocation will grow the heap with the big amount. This diff
> > changes the strategy by not re-using a block from the free list if
> > half the space or more would be wasted. Instead, it grows the heap by
> > the requested amount.
> >
> > This make it possible for me to boot using a root fs with a large
> > blocksize. There have been several reports of large roots not working
> > (the bootloader allocates memory based om the blocksize of the file
> > system, and by default larger filesystems use larger blocks).
> >
> > How to test
> > ===========
> >
> > Apply diff and do a full build including building release. After that,
> > either upgrade using your newly built cd64.iso, bsd.rd or other
> > mechanism or do a full install. Test that you can boot afterwards.
> >
> > This needs to be tested on various platforms, both will small and big
> > (> 600G) root filesystems. Yes, this is tedious, but we want large
> > coverage of different cases.
> >
> > -Otto
>
> As it turns out by my own testing, on amd64 root filssytems using 32k
> blocks now work fine, but 64k fs blocks still hit a ceiling. This
> corresponds to > 512G disks if you use the defaults.
>
> -Otto
>
New diff that also works on root filesystems > 500G. It avoid using a
large bouncebuffer by reding large buffers in a loop instead of one go.
-Otto
Index: arch/amd64/stand/libsa/biosdev.c
===================================================================
RCS file: /cvs/src/sys/arch/amd64/stand/libsa/biosdev.c,v
retrieving revision 1.32
diff -u -p -r1.32 biosdev.c
--- arch/amd64/stand/libsa/biosdev.c 10 Aug 2018 16:41:35 -0000 1.32
+++ arch/amd64/stand/libsa/biosdev.c 11 Dec 2018 13:00:02 -0000
@@ -340,11 +340,26 @@ biosd_io(int rw, bios_diskinfo_t *bd, u_
return error;
}
+#define MAXSECTS 32
+
int
biosd_diskio(int rw, struct diskinfo *dip, u_int off, int nsect, void *buf)
{
- return biosd_io(rw, &dip->bios_info, off, nsect, buf);
+ int i, n, ret;
+
+ /*
+ * Avoid doing too large reads, the bounce buffer used by biosd_io()
+ * might run us out-of-mem.
+ */
+ for (i = 0, ret = 0; ret == 0 && nsect > 0;
+ i += MAXSECTS, nsect -= MAXSECTS) {
+ n = nsect >= MAXSECTS ? MAXSECTS : nsect;
+ ret = biosd_io(rw, &dip->bios_info, off + i, n,
+ buf + i * DEV_BSIZE);
+ }
+ return ret;
}
+
/*
* Try to read the bsd label on the given BIOS device.
*/
@@ -715,7 +730,6 @@ biosstrategy(void *devdata, int rw, dadd
size_t *rsize)
{
struct diskinfo *dip = (struct diskinfo *)devdata;
- bios_diskinfo_t *bd = &dip->bios_info;
u_int8_t error = 0;
size_t nsect;
@@ -732,7 +746,7 @@ biosstrategy(void *devdata, int rw, dadd
if (blk < 0)
error = EINVAL;
else
- error = biosd_io(rw, bd, blk, nsect, buf);
+ error = biosd_diskio(rw, dip, blk, nsect, buf);
#ifdef BIOS_DEBUG
if (debug) {
Index: arch/i386/stand/libsa/biosdev.c
===================================================================
RCS file: /cvs/src/sys/arch/i386/stand/libsa/biosdev.c,v
retrieving revision 1.98
diff -u -p -r1.98 biosdev.c
--- arch/i386/stand/libsa/biosdev.c 6 Sep 2018 11:50:54 -0000 1.98
+++ arch/i386/stand/libsa/biosdev.c 11 Dec 2018 13:00:02 -0000
@@ -341,11 +341,26 @@ biosd_io(int rw, bios_diskinfo_t *bd, u_
return error;
}
+#define MAXSECTS 32
+
int
biosd_diskio(int rw, struct diskinfo *dip, u_int off, int nsect, void *buf)
{
- return biosd_io(rw, &dip->bios_info, off, nsect, buf);
+ int i, n, ret;
+
+ /*
+ * Avoid doing too large reads, the bounce buffer used by biosd_io()
+ * might run us out-of-mem.
+ */
+ for (i = 0, ret = 0; ret == 0 && nsect > 0;
+ i += MAXSECTS, nsect -= MAXSECTS) {
+ n = nsect >= MAXSECTS ? MAXSECTS : nsect;
+ ret = biosd_io(rw, &dip->bios_info, off + i, n,
+ buf + i * DEV_BSIZE);
+ }
+ return ret;
}
+
/*
* Try to read the bsd label on the given BIOS device.
*/
@@ -716,7 +731,6 @@ biosstrategy(void *devdata, int rw, dadd
size_t *rsize)
{
struct diskinfo *dip = (struct diskinfo *)devdata;
- bios_diskinfo_t *bd = &dip->bios_info;
u_int8_t error = 0;
size_t nsect;
@@ -733,7 +747,7 @@ biosstrategy(void *devdata, int rw, dadd
if (blk < 0)
error = EINVAL;
else
- error = biosd_io(rw, bd, blk, nsect, buf);
+ error = biosd_diskio(rw, dip, blk, nsect, buf);
#ifdef BIOS_DEBUG
if (debug) {
Index: lib/libsa/alloc.c
===================================================================
RCS file: /cvs/src/sys/lib/libsa/alloc.c,v
retrieving revision 1.12
diff -u -p -r1.12 alloc.c
--- lib/libsa/alloc.c 14 Mar 2016 23:08:06 -0000 1.12
+++ lib/libsa/alloc.c 11 Dec 2018 13:00:03 -0000
@@ -169,7 +169,7 @@ alloc(unsigned int size)
}
/* no match in freelist if bestsize unchanged */
- failed = (bestsize == 0xffffffff);
+ failed = (bestsize == 0xffffffff || bestsize >= size * 2);
#endif
if (failed) { /* nothing found */
