Re: [PATCH 0/10]: Respin of sparc patches.
On Fri, Mar 06, 2009 at 04:38:47PM -0800, David Miller wrote: From: Robert Millan r...@aybabtu.com Date: Fri, 6 Mar 2009 20:01:10 +0100 In the future, please send followup patches in the same thread. It helps keeping track of things, which is quite hard for us to keep up with already.. Sure no problem. What's going to happen with these patches? Well, we need time both for the paperwork and for reviewing them. I'm not sure how many of us are experienced with sparc (Vincent? Who else?). I at least can comment on Ieee1275 stuff, so does Pavel, Marco, Manoel and probably others. We'll get there. Just be patient :-) I am able to boot into the GRUB kernel prompt on my test system, I have a lot of new code written, I'm _EXTREMELY_ motivated and want to get this all working fully in the next week or so. I'm very interested to see proper sparc support completed/merged. Your effort is much appreciated. -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 1/7]: Fix sparc64 setjmp implementation.
On Sat, Mar 07, 2009 at 12:56:43AM -0500, Pavel Roskin wrote: Quoting David Miller da...@davemloft.net: I was considering making grub_size_t long and grub_ssize_t unsigned long. I remember that it required many changes in string formats, so I didn't feel it would be justified. But we could try it again. You should use whatever is the appropriate size_t/ssize_t type on the given platform+ABI, and that way GRUB could use %Zd and/or %zd throughout. Was that the plan? No, the plan was to use long. We don't support architectures where long and size_t have different sizes. I'm not aware of such architectures. I don't care about win64, as it would need some very special handling (perhaps redefinition of long). Maybe Christian can shred some light on this ;-) -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH] make the generic Linux loader usable on i386-pc again
On Fri, Mar 06, 2009 at 10:02:28PM +0200, Vesa Jääskeläinen wrote: Robert Millan wrote: This patch makes the generic Linux loader usable on i386-pc again. It doesn't seem like it's badly needed to spend a bit of time and a bit of code in adding low memory to the heap, and Vesa's work on the new memory manager should give a proper solution to this problem. I think in the meantime we could just not allocate low mem, assuming nobody has a problem with that. If this really blocks it I have nothing against it. But could you share a bit insight what kind of memories are required to be where for Linux? Sure. Note that my experience is merely derived from our existing code (which I had to in order to produce the initial loader/i386/linux.c). Linux developers reading this (hi Dave ;-)) probably know better. There's an area between 0x1 and 0x9 which is where struct linux_kernel_params needs to be stored. This includes a statically allocated memory map. There's an area between 0x10 and min(0x37FF,(grub_os_area_addr + grub_os_area_size)) where the actual Linux image is loaded, as well as the initrd (inmediately after it, with some alignment). I think that's all. -- Robert Millan The DRM opt-in fallacy: Your data belongs to us. We will decide when (and how) you may access your data; but nobody's threatening your freedom: we still allow you to remove your data and not access it at all. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: Parser
Kevin Lacquement ke...@lacqui.com writes: Again looking at the script parser, I notices that it uses a Yacc-generated parser, but a hand-written tokenizer. Is there a reason that it doesn't use Lex? Is it due to external dependencies, and if so, is there a way to recreate these deps (library or whatever) within the constraints of the bootloader? Right, and I really hate the handwritten parser. It is easy to cleanly use bison. Lex, OTOH, isn't easy and clean to use. IIRC it depended on file IO, etc. If you know a *clean* way to deal with this, I would favor it. Making all kinds of dirty stubs is something I don't really like. -- Marco ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH] make the generic Linux loader usable on i386-pc again
Small word about xnu. It needs for norma booting just a quite big amount of contiguos memory anywhere below 4GiB. This is typically 64 MiB depending on the amount of loaded modules but can be much larger (e.g. if using ramdisk) In the case of resuming from hibernation it needs small amount at 0x10 (around 64 kib probably) and a space for compressed hibernation image. Depending on system usage prior to hibernation it could be as big as the whole memory minus few MiB. Robert Millan wrote: On Fri, Mar 06, 2009 at 10:02:28PM +0200, Vesa Jääskeläinen wrote: Robert Millan wrote: This patch makes the generic Linux loader usable on i386-pc again. It doesn't seem like it's badly needed to spend a bit of time and a bit of code in adding low memory to the heap, and Vesa's work on the new memory manager should give a proper solution to this problem. I think in the meantime we could just not allocate low mem, assuming nobody has a problem with that. If this really blocks it I have nothing against it. But could you share a bit insight what kind of memories are required to be where for Linux? Sure. Note that my experience is merely derived from our existing code (which I had to in order to produce the initial loader/i386/linux.c). Linux developers reading this (hi Dave ;-)) probably know better. There's an area between 0x1 and 0x9 which is where struct linux_kernel_params needs to be stored. This includes a statically allocated memory map. There's an area between 0x10 and min(0x37FF,(grub_os_area_addr + grub_os_area_size)) where the actual Linux image is loaded, as well as the initrd (inmediately after it, with some alignment). I think that's all. -- Regards Vladimir 'phcoder' Serbinenko ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 0/10]: Respin of sparc patches.
From: Robert Millan r...@aybabtu.com Date: Sat, 7 Mar 2009 11:37:36 +0100 On Fri, Mar 06, 2009 at 04:38:47PM -0800, David Miller wrote: I am able to boot into the GRUB kernel prompt on my test system, I have a lot of new code written, I'm _EXTREMELY_ motivated and want to get this all working fully in the next week or so. I'm very interested to see proper sparc support completed/merged. Your effort is much appreciated. FWIW, everything other than actually booting a Linux kernel is functional in my local tree. This includes normal mode, bouncing between that and rescue mode, loading modules, listing devices and files on ext2 filesystems, etc. It all works. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 1/7]: Fix sparc64 setjmp implementation.
From: Robert Millan r...@aybabtu.com Date: Sat, 7 Mar 2009 11:38:38 +0100 On Sat, Mar 07, 2009 at 12:56:43AM -0500, Pavel Roskin wrote: Quoting David Miller da...@davemloft.net: I was considering making grub_size_t long and grub_ssize_t unsigned long. I remember that it required many changes in string formats, so I didn't feel it would be justified. But we could try it again. You should use whatever is the appropriate size_t/ssize_t type on the given platform+ABI, and that way GRUB could use %Zd and/or %zd throughout. Was that the plan? No, the plan was to use long. We don't support architectures where long and size_t have different sizes. I'm not aware of such architectures. I don't care about win64, as it would need some very special handling (perhaps redefinition of long). Maybe Christian can shred some light on this ;-) I've checked a lot of cases (other than win64) and this assumption that sizeof(long)==sizeof(size_t) seems to hold. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
[Design] nested partitions: Unify grub_partition and grub_disk
Hello. I was looking into nested partitions issue and propose to unify grub_partition and grub_disk. In this case when a code calls grub_disk_read with a grub_disk_t which represents a partition it will correct the offset and call grub_disk_read with underlying partition or disk. In this way not only nested partitions become natural but also conceptually similar fields of total_sectors and len become unified. As it's an important design change I send this e-mail before coding it -- Regards Vladimir 'phcoder' Serbinenko ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [Design] nested partitions: Unify grub_partition and grub_disk
I forgot to speak about another question: partition naming. I see 2 possibilities 1) purely numeric unified naming scheme. It means that (hd0,1,a) becomes (hd0,1,1) On one hand mixed number-letter scheme is similar to what freebsd uses but on the other hand numerical scheme is versatile and allows unlimited nestedness. And I don't see why we would use a scheme specific to one of many supported OSes. 2) Every partition map is allowed to pick the name that it likes as long as it contains no comma. In this way we would need to keep partition-name parsing functions in partitition map modules. It means that this code would be duplicated. But this scheme is better in the cases when partition map has no numbering scheme but instead has labels attached to partitions. But in this case IMO search command should be used find the partition I personally would prefer the first way Also an interesting question is how would has_partitions field be handled in this scheme. Just ignored. It's actually used only to optimise some code out based on the assumption that some media has no partitions. Performance gain is negligible but if this assumption doesn't hold true grub won't be able to access the partitions which are really here. Famous example is a cdrom. Most people would assume that cdrom has no partitions. But on powerpc bootable cdroms use APM -- Regards Vladimir 'phcoder' Serbinenko ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [Design] nested partitions: Unify grub_partition and grub_disk
Hi, Hello. I was looking into nested partitions issue and propose to unify grub_partition and grub_disk. In this case when a code calls grub_disk_read with a grub_disk_t which represents a partition it will correct the offset and call grub_disk_read with underlying partition or disk. In this way not only nested partitions become natural but also conceptually similar fields of total_sectors and len become unified. As it's an important design change I send this e-mail before coding it Actually, this is already how it works. grub_disk has a partition pointer, which indicate if it is partition or disk, and grub_disk_adjust_range adjust the offset for partition. My suggestion is, always treats grub_disk as a partition, and use the offset. For the root partition, offset is 0 so we get the whole disk. On Sun, Mar 8, 2009 at 12:03 AM, phcoder phco...@gmail.com wrote: I forgot to speak about another question: partition naming. I see 2 possibilities 1) purely numeric unified naming scheme. It means that (hd0,1,a) becomes (hd0,1,1) On one hand mixed number-letter scheme is similar to what freebsd uses but on the other hand numerical scheme is versatile and allows unlimited nestedness. And I don't see why we would use a scheme specific to one of many supported OSes. 2) Every partition map is allowed to pick the name that it likes as long as it contains no comma. In this way we would need to keep partition-name parsing functions in partitition map modules. It means that this code would be duplicated. But this scheme is better in the cases when partition map has no numbering scheme but instead has labels attached to partitions. But in this case IMO search command should be used find the partition I personally would prefer the first way I'm fine with names like (hd0,1,1). Also an interesting question is how would has_partitions field be handled in this scheme. Just ignored. It's actually used only to optimise some code out based on the assumption that some media has no partitions. Performance gain is negligible but if this assumption doesn't hold true grub won't be able to access the partitions which are really here. Famous example is a cdrom. Most people would assume that cdrom has no partitions. But on powerpc bootable cdroms use APM This flag could be useful sometimes. For example, in raid devices, the first disk may contain partition table, but we shouldn't use it. -- Bean ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [Design] nested partitions: Unify grub_partition and grub_disk
Bean wrote: Hi, Hello. I was looking into nested partitions issue and propose to unify grub_partition and grub_disk. In this case when a code calls grub_disk_read with a grub_disk_t which represents a partition it will correct the offset and call grub_disk_read with underlying partition or disk. In this way not only nested partitions become natural but also conceptually similar fields of total_sectors and len become unified. As it's an important design change I send this e-mail before coding it Actually, this is already how it works. grub_disk has a partition pointer, which indicate if it is partition or disk, and grub_disk_adjust_range adjust the offset for partition. My suggestion is, always treats grub_disk as a partition, and use the offset. For the root partition, offset is 0 so we get the whole disk. The problem with such a design is that the partition code would always have to add parent partition offset to the start of subpartitions. It's something that I would like to hide from partition code mainly because people don't usually care if their code works well in nested partition environments and its cleaner if partition code doesn't need to not whether it divides a disk or subdivides a partition. In my proposition at each step I call a function which naturally resolves the current level of indirection. E.g. if we have sun partition in fdisk partition on biosdisk I call: 1) device-partition-probe it gives the offset 2) device-parent-partition-probe it gives the second offset 3) device-parent-parent-dev-read As you see at every step I just resolve a level of indirection This flag could be useful sometimes. For example, in raid devices, the first disk may contain partition table, but we shouldn't use it. I'm not familiar with raid but could it be that boot partition is at the first disk of raid just after the partition map. Then to read it we don't need raid modules. I don't see why such a usage case should be specifically excluded And don't raid disk contain a volume header? If they do how can they contain the partition table in the usual place Also existence of (hd0,1) doesn't prevent you from accessing (hd0) -- Regards Vladimir 'phcoder' Serbinenko ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: [PATCH 0/10]: Respin of sparc patches.
Le Saturday 07 March 2009 11:37:36 Robert Millan, vous avez écrit : I'm not sure how many of us are experienced with sparc (Vincent? Who else?). I wouldn't call me experienced, except if it only means that I did experimentation with that arch :) . I'm very interested to see proper sparc support completed/merged. Your effort is much appreciated. +1 -- Vincent Pelletier signature.asc Description: This is a digitally signed message part. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
[PATCH] dynamic cache allocation
Hello. Discussing with Robert Millan and Bean on IRC we noticed that
disk cache index is statically allocated. Here is a proposal to change
it to dynamic allocation proportional to the size of available memory.
--
Regards
Vladimir 'phcoder' Serbinenko
Index: kern/disk.c
===
--- kern/disk.c (revision 2021)
+++ kern/disk.c (working copy)
@@ -41,7 +41,8 @@ struct grub_disk_cache
int lock;
};
-static struct grub_disk_cache grub_disk_cache_table[GRUB_DISK_CACHE_NUM];
+static struct grub_disk_cache *grub_disk_cache_table = 0;
+static int grub_disk_cache_num = 0;
void (*grub_disk_firmware_fini) (void);
int grub_disk_firmware_is_tainted;
@@ -68,7 +69,7 @@ grub_disk_cache_get_index (unsigned long dev_id, u
{
return ((dev_id * 524287UL + disk_id * 2606459UL
+ ((unsigned) (sector GRUB_DISK_CACHE_BITS)))
- % GRUB_DISK_CACHE_NUM);
+ % grub_disk_cache_num);
}
static void
@@ -78,6 +79,9 @@ grub_disk_cache_invalidate (unsigned long dev_id,
unsigned index;
struct grub_disk_cache *cache;
+ if (!grub_disk_cache_table)
+return;
+
sector = ~(GRUB_DISK_CACHE_SIZE - 1);
index = grub_disk_cache_get_index (dev_id, disk_id, sector);
cache = grub_disk_cache_table + index;
@@ -97,7 +101,10 @@ grub_disk_cache_invalidate_all (void)
{
unsigned i;
- for (i = 0; i GRUB_DISK_CACHE_NUM; i++)
+ if (!grub_disk_cache_table)
+return;
+
+ for (i = 0; i grub_disk_cache_num; i++)
{
struct grub_disk_cache *cache = grub_disk_cache_table + i;
@@ -116,6 +123,14 @@ grub_disk_cache_fetch (unsigned long dev_id, unsig
struct grub_disk_cache *cache;
unsigned index;
+ if (!grub_disk_cache_table)
+{
+#if 0
+ grub_disk_cache_misses++;
+#endif
+ return 0;
+}
+
index = grub_disk_cache_get_index (dev_id, disk_id, sector);
cache = grub_disk_cache_table + index;
@@ -143,6 +158,9 @@ grub_disk_cache_unlock (unsigned long dev_id, unsi
struct grub_disk_cache *cache;
unsigned index;
+ if (!grub_disk_cache_table)
+return ;
+
index = grub_disk_cache_get_index (dev_id, disk_id, sector);
cache = grub_disk_cache_table + index;
@@ -157,7 +175,22 @@ grub_disk_cache_store (unsigned long dev_id, unsig
{
unsigned index;
struct grub_disk_cache *cache;
-
+
+ if (!grub_disk_cache_table)
+{
+#ifdef GRUB_UTIL
+ grub_disk_cache_num = GRUB_UTIL_CACHE_NUM;
+#else
+ grub_disk_cache_num = grub_mm_get_free () / GRUB_DISK_CACHE_DIVIDE;
+#endif
+ grub_disk_cache_table = (struct grub_disk_cache *)
+ grub_malloc (grub_disk_cache_num * sizeof (struct grub_disk_cache));
+}
+
+ if (!grub_disk_cache_table)
+return grub_error (GRUB_ERR_OUT_OF_MEMORY,
+ couldn't allocate space for cache index);
+
grub_disk_cache_invalidate (dev_id, disk_id, sector);
index = grub_disk_cache_get_index (dev_id, disk_id, sector);
Index: kern/mm.c
===
--- kern/mm.c (revision 2021)
+++ kern/mm.c (working copy)
@@ -449,6 +449,33 @@ grub_realloc (void *ptr, grub_size_t size)
return q;
}
+grub_size_t
+grub_mm_get_free (void)
+{
+ grub_mm_region_t r;
+ grub_size_t ret = 0;
+
+ for (r = base; r; r = r-next)
+{
+ grub_mm_header_t p;
+
+ /* Follow the free list. */
+ p = r-first;
+ do
+ {
+ if (p-magic != GRUB_MM_FREE_MAGIC)
+ grub_fatal (free magic is broken at %p: 0x%x, p, p-magic);
+
+ ret += p-size GRUB_MM_ALIGN_LOG2;
+ p = p-next;
+ }
+ while (p != r-first);
+}
+
+ return ret;
+}
+
+
#ifdef MM_DEBUG
int grub_mm_debug = 0;
Index: include/grub/disk.h
===
--- include/grub/disk.h (revision 2021)
+++ include/grub/disk.h (working copy)
@@ -129,9 +129,14 @@ typedef struct grub_disk_memberlist *grub_disk_mem
#define GRUB_DISK_SECTOR_SIZE 0x200
#define GRUB_DISK_SECTOR_BITS 9
-/* The maximum number of disk caches. */
-#define GRUB_DISK_CACHE_NUM 1021
+/* Allocate one cache entry per N bytes.
+ By default use up to one quarter of memory available for allocation */
+#define GRUB_DISK_CACHE_DIVIDE 16536
+/* On grub-util we can't know the available memory size and since host OS
+ already caches data we don't need a big cache*/
+#define GRUB_UTIL_CACHE_NUM 100
+
/* The size of a disk cache in sector units. */
#define GRUB_DISK_CACHE_SIZE 8
#define GRUB_DISK_CACHE_BITS 3
Index: include/grub/mm.h
===
--- include/grub/mm.h (revision 2021)
+++ include/grub/mm.h (working copy)
@@ -33,6 +33,7 @@ void *EXPORT_FUNC(grub_malloc) (grub_size_t size);
void EXPORT_FUNC(grub_free) (void *ptr);
void *EXPORT_FUNC(grub_realloc) (void *ptr, grub_size_t size);
void *EXPORT_FUNC(grub_memalign) (grub_size_t align, grub_size_t size);
+grub_size_t EXPORT_FUNC(grub_mm_get_free) (void);
/* For debugging. */
#if defined(MM_DEBUG)
