Re: [PATCH] Move NTFS compression support to new module ntfscomp
Committed. On Sat, Dec 15, 2007 at 02:08:22PM +0100, Robert Millan wrote: I just tested this with and without the module, on a filesystem with and without compression (all 4 combinations) and didn't observe any oddities. If nobody objects I'll check this in. On Wed, Dec 12, 2007 at 11:56:36PM +0800, Bean wrote: Ok. 2007-12-12 Bean [EMAIL PROTECTED] * conf/common.rmk (pkgdata_MODULES): Add ntfscomp.mod. (ntfscomp_mod_SOURCES): New variable. (ntfscomp_mod_CFLAGS): Likewise. (ntfscomp_mod_LDFLAGS): Likewise. * conf/i386-pc.rmk (grub_setup_SOURCES): Add fs/ntfscomp.c. (grub_probe_SOURCES): Likewise. (grub_emu_SOURCES): Likewise. * conf/i386-efi.rmk (grub_probe_SOURCES): Add fs/ntfscomp.c. (grub_emu_SOURCES): Likewise. * conf/i386-linuxbios.rmk (grub_probe_SOURCES): Add fs/ntfscomp.c. (grub_emu_SOURCES): Likewise. * conf/powerpc-ieee1275.rmk (grub_probe_SOURCES): Add fs/ntfscomp.c. (grub_emu_SOURCES): Likewise. * fs/ntfs.c (grub_ntfscomp_func): New variable. (read_run_list): Renamed to grub_ntfs_read_run_list. (decomp_nextvcn): Moved to ntfscomp.c. (decomp_getch): Likewise. (decomp_get16): Likewise. (decomp_block): Likewise. (read_block): Likewise. (read_data): Partially moved to ntfscomp.c. (fixup): Change unsigned to grub_uint16_t. (read_mft): Change unsigned long to grub_uint32_t. (read_attr): Likewise. (read_data): Likewise. (read_run_data): Likewise. (read_run_list): Likewise. (read_mft): Likewise. * fs/ntfscomp.c: New file. * include/grub/ntfs.h: New file. Index: conf/common.rmk === RCS file: /sources/grub/grub2/conf/common.rmk,v retrieving revision 1.17 diff -u -p -r1.17 common.rmk --- conf/common.rmk 18 Nov 2007 06:41:45 - 1.17 +++ conf/common.rmk 29 Nov 2007 18:23:10 - @@ -81,7 +81,7 @@ update-grub_DATA += util/grub.d/README # Filing systems. pkgdata_MODULES += fshelp.mod fat.mod ufs.mod ext2.mod ntfs.mod\ minix.mod hfs.mod jfs.mod iso9660.mod xfs.mod affs.mod \ - sfs.mod hfsplus.mod + sfs.mod hfsplus.mod ntfscomp.mod # For fshelp.mod. fshelp_mod_SOURCES = fs/fshelp.c @@ -108,6 +108,11 @@ ntfs_mod_SOURCES = fs/ntfs.c ntfs_mod_CFLAGS = $(COMMON_CFLAGS) ntfs_mod_LDFLAGS = $(COMMON_LDFLAGS) +# For ntfscomp.mod. +ntfscomp_mod_SOURCES = fs/ntfscomp.c +ntfscomp_mod_CFLAGS = $(COMMON_CFLAGS) +ntfscomp_mod_LDFLAGS = $(COMMON_LDFLAGS) + # For minix.mod. minix_mod_SOURCES = fs/minix.c minix_mod_CFLAGS = $(COMMON_CFLAGS) Index: conf/i386-efi.rmk === RCS file: /sources/grub/grub2/conf/i386-efi.rmk,v retrieving revision 1.24 diff -u -p -r1.24 i386-efi.rmk --- conf/i386-efi.rmk 18 Nov 2007 06:41:45 - 1.24 +++ conf/i386-efi.rmk 29 Nov 2007 18:23:11 - @@ -36,7 +36,7 @@ grub_probe_SOURCES = util/grub-probe.c\ partmap/pc.c partmap/apple.c partmap/gpt.c \ fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c fs/ntfs.c kern/fs.c \ kern/env.c fs/fshelp.c fs/xfs.c fs/affs.c fs/sfs.c fs/hfsplus.c \ - disk/lvm.c disk/raid.c grub_probe_init.c + fs/ntfscomp.c disk/lvm.c disk/raid.c grub_probe_init.c # For grub-emu. grub_emu_DEPENDENCIES = grub_script.tab.c grub_script.tab.h \ @@ -50,7 +50,7 @@ grub_emu_SOURCES = commands/boot.c comma disk/loopback.c \ fs/affs.c fs/ext2.c fs/fat.c fs/fshelp.c fs/hfs.c fs/iso9660.c \ fs/jfs.c fs/minix.c fs/sfs.c fs/ufs.c fs/xfs.c fs/hfsplus.c \ - fs/ntfs.c \ + fs/ntfs.c fs/ntfscomp.c \ io/gzio.c \ kern/device.c kern/disk.c kern/dl.c kern/elf.c kern/env.c \ kern/err.c \ Index: conf/i386-linuxbios.rmk === RCS file: /sources/grub/grub2/conf/i386-linuxbios.rmk,v retrieving revision 1.3 diff -u -p -r1.3 i386-linuxbios.rmk --- conf/i386-linuxbios.rmk 18 Nov 2007 06:41:45 - 1.3 +++ conf/i386-linuxbios.rmk 29 Nov 2007 18:23:14 - @@ -60,7 +60,7 @@ grub_probe_SOURCES = util/grub-probe.c\ partmap/pc.c partmap/apple.c partmap/gpt.c \ fs/ufs.c fs/minix.c fs/hfs.c fs/jfs.c fs/ntfs.c kern/fs.c \ kern/env.c fs/fshelp.c fs/xfs.c fs/affs.c fs/sfs.c fs/hfsplus.c \ - disk/lvm.c disk/raid.c grub_probe_init.c + fs/ntfscomp.c disk/lvm.c disk/raid.c grub_probe_init.c # For grub-emu. grub_emu_DEPENDENCIES =
Re: moving ata initialisation to a command
Robert Millan [EMAIL PROTECTED] writes: On Mon, Dec 17, 2007 at 01:26:31PM -0500, Pavel Roskin wrote: On Mon, 2007-12-17 at 17:01 +0100, Marco Gerards wrote: A better solution, IMO, would be changing grub-mkrescue so it doesn't load all modules. Maybe grub-mkrescue should create a filesystem? Even FAT should be fine. This way, it will be possible to load problematic modules from the filesystem. The only problem would be dependency on filesystem making tools. Fortunately, mtools is quite common. I'd prefer to support embedding filesystems in core.img instead. This solves the problem for every situation instead of just PC boot media (e.g. LinuxBIOS ELF payload image). Agreed, I would like to see such feature. Also for stuff like diskless boot. What I had in mind was the following. We could perhaps make it possible to add all files using grub-mkimage, not only modules. When you add modules, those will be loaded automatically. All other files will be ignored. Another filesystem module has to be written. It scans over these files and will make them available. This can be simple and elegant. I do not like the idea of adding a diskimage or so. Are you interested in working on this? Anyway, as long as root privileges aren't made necessary, I'm fine with grub-mkrescue creating a filesystem (one saner than FAT, preferably ;-). Great! Will you do that? :-) -- Marco ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: Switching to git?
Pavel Roskin wrote: On Tue, 2007-12-18 at 20:30 +0200, Vesa Jääskeläinen wrote: Just leave cygwin out of the box... thank you! cygwin is one of the worst pieces of software that just does not work correctly. I wonder if you have actually used the native Windows port of CVS. Do you want my copy of it ? CVS is quite freely available as native version. And what the better, CVS integrates quite nicely to Eclipse. Same goes for Subversion too. In my point-of-view portability means that you can use software natively on some platform and it does not include installing emulators or such to run software (like cygwin, wine). Speaking of Cygwin, my experience with it has been much more positive, especially if it's installed for one specific purpose like compiling some project (as opposed to a playground for UNIX wannabees). I don't think it's a requirement that is going to deter anyone from working on GRUB. You answered it yourself ;) IT just doesn't work for doing multiple things. You have to carefully keep settings in working order if you want to use it for multiple projects. Ever tried to compile Mozilla Thunderbird under Windows? And if you need any GUI, qgit is a native Windows application compiled with Qt4. The Tortoise like GUI doesn't make much sense for git, as it's file oriented, and git is changeset oriented. The Eclipse plugin (egit) is available. And have you really used the egit ? Eg. how well it works and so on. I prefer not to jump between programs... It is just awkward. ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: moving ata initialisation to a command
On Fri, Dec 21, 2007 at 05:47:36PM +0100, Marco Gerards wrote: Robert Millan [EMAIL PROTECTED] writes: On Mon, Dec 17, 2007 at 01:26:31PM -0500, Pavel Roskin wrote: On Mon, 2007-12-17 at 17:01 +0100, Marco Gerards wrote: A better solution, IMO, would be changing grub-mkrescue so it doesn't load all modules. Maybe grub-mkrescue should create a filesystem? Even FAT should be fine. This way, it will be possible to load problematic modules from the filesystem. The only problem would be dependency on filesystem making tools. Fortunately, mtools is quite common. I'd prefer to support embedding filesystems in core.img instead. This solves the problem for every situation instead of just PC boot media (e.g. LinuxBIOS ELF payload image). Agreed, I would like to see such feature. Also for stuff like diskless boot. What I had in mind was the following. We could perhaps make it possible to add all files using grub-mkimage, not only modules. When you add modules, those will be loaded automatically. All other files will be ignored. Another filesystem module has to be written. It scans over these files and will make them available. This can be simple and elegant. I do not like the idea of adding a diskimage or so. Are you interested in working on this? Well, I already have an almost-working implementation. I guess I'll have to defend it then! :-P Really, it's clean and elegant. Very simple and flexible. Once I get the address detection right, that is.. Anyway, as long as root privileges aren't made necessary, I'm fine with grub-mkrescue creating a filesystem (one saner than FAT, preferably ;-). Great! Will you do that? :-) No, I tried once and it was a mess of things. Besides, when we have embedded filesystem support this won't add any real advantage. -- Robert Millan GPLv2 I know my rights; I want my phone call! DRM What use is a phone call, if you are unable to speak? (as seen on /.) ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: Creating grub/grub2/grldr.mbr bootrom with makerom
On Sat, Dec 22, 2007 at 02:50:11AM +0800, Bean wrote: makerom is a new tool in the GRUB Utilities project. It can be used to create bootrom image using grub legacy, grub2 and grldr.mbr. grub legacy/grub4dos: makerom rom_xxx.img stage2 grub.bin grub2: makerom rom_xxx.img core.img grub2.bin grldr.mbr: makerom -s 0 rom_xxx.img grldr.mbr grldr.bin rom_xxx.img stands for one of the following files: rom_isa.img: Header for ISA bootrom rom_pci.img: Header for PCI bootrom rom_zisa.img: Header for ISA bootrom, support lzma decompression rom_zpci.img: Header for PCI bootrom, support lzma decompression To enable lzma, you should use option -z, and choose one of the headers that supports lzma: makerom -z rom_zisa.img stage2 grub.bin As bootrom has 64K size limit, for image larger than 64K, you must use compression. For example, the uncompressed stage2 for grub 0.97 is 98K, after compression, image reduced to about 50K. Also note that compression is performed by external lzma program, you need to install the lzma package before using the compression function. How well does compression work for GRUB 2 ? core.img is already compressed (with lzo); if LZMA makes better results perhaps it'd be a good idea to switch. -- Robert Millan GPLv2 I know my rights; I want my phone call! DRM What use is a phone call, if you are unable to speak? (as seen on /.) ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Creating grub/grub2/grldr.mbr bootrom with makerom
makerom is a new tool in the GRUB Utilities project. It can be used to create bootrom image using grub legacy, grub2 and grldr.mbr. grub legacy/grub4dos: makerom rom_xxx.img stage2 grub.bin grub2: makerom rom_xxx.img core.img grub2.bin grldr.mbr: makerom -s 0 rom_xxx.img grldr.mbr grldr.bin rom_xxx.img stands for one of the following files: rom_isa.img: Header for ISA bootrom rom_pci.img: Header for PCI bootrom rom_zisa.img: Header for ISA bootrom, support lzma decompression rom_zpci.img: Header for PCI bootrom, support lzma decompression To enable lzma, you should use option -z, and choose one of the headers that supports lzma: makerom -z rom_zisa.img stage2 grub.bin As bootrom has 64K size limit, for image larger than 64K, you must use compression. For example, the uncompressed stage2 for grub 0.97 is 98K, after compression, image reduced to about 50K. Also note that compression is performed by external lzma program, you need to install the lzma package before using the compression function. To test the bootrom in qemu: qemu -option-rom grub.bin ... Other options supported by makerom: -m message The message to display. -t timeout Timeout in seconds, default value is 2. -k key Hot key, default value SPACE. -d edx The value of %edx registry, default value 0. -s skip Number of bytes to skip at the beginning of input file, default value 512, which is correct for stage2 and core.img. For grldr.mbr, you should set this value to 0. -z Use lzma to compress input file. -p parm Extra options for lzma, it can be used to adjust compression ratio. The program to invoke is lzma in windows, and lzma_alone in Linux. The default command line is: lzma_alone e -si -so -v Show verbose information. You can download source code and windows binaries at: http://download.gna.org/grubutil/ -- Bean ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
embedding filesystems in core image (Re: moving ata initialisation to a command)
On Fri, Dec 21, 2007 at 07:04:05PM +0100, Robert Millan wrote:
I'd prefer to support embedding filesystems in core.img instead. This
solves the problem for every situation instead of just PC boot media
(e.g. LinuxBIOS ELF payload image).
Agreed, I would like to see such feature. Also for stuff like
diskless boot.
What I had in mind was the following. We could perhaps make it
possible to add all files using grub-mkimage, not only modules. When
you add modules, those will be loaded automatically. All other files
will be ignored.
Another filesystem module has to be written. It scans over these
files and will make them available. This can be simple and elegant.
I do not like the idea of adding a diskimage or so. Are you
interested in working on this?
Well, I already have an almost-working implementation. I guess I'll have to
defend it then! :-P
Really, it's clean and elegant. Very simple and flexible. Once I get the
address detection right, that is..
Please have a look at the attached patch. This allows adding an arbitrary
number of files. No extra filesystem module is needed, since any of the
supported filesystems can be used with this feature:
genisoimage -o filesystem_image -r stuff
grub-mkimage -o core.img -m filesystem_image
--
Robert Millan
GPLv2 I know my rights; I want my phone call!
DRM What use is a phone call, if you are unable to speak?
(as seen on /.)
diff -x '*.mk' -Nurp grub2/conf/i386-pc.rmk grub2.memdisk/conf/i386-pc.rmk
--- grub2/conf/i386-pc.rmk 2007-12-21 12:41:23.0 +0100
+++ grub2.memdisk/conf/i386-pc.rmk 2007-12-21 21:47:57.0 +0100
@@ -39,7 +39,7 @@ kernel_img_HEADERS = arg.h boot.h cache.
env.h err.h file.h fs.h kernel.h loader.h misc.h mm.h net.h parser.h \
partition.h pc_partition.h rescue.h symbol.h term.h time.h types.h \
machine/biosdisk.h machine/boot.h machine/console.h machine/init.h \
- machine/memory.h machine/loader.h machine/vga.h machine/vbe.h
+ machine/memory.h machine/loader.h machine/vga.h machine/vbe.h machine/kernel.h
kernel_img_CFLAGS = $(COMMON_CFLAGS)
kernel_img_ASFLAGS = $(COMMON_ASFLAGS)
kernel_img_LDFLAGS = $(COMMON_LDFLAGS) -Wl,-N,-Ttext,8200 $(COMMON_CFLAGS)
@@ -136,7 +136,7 @@ pkgdata_MODULES = biosdisk.mod _chain.mo
_multiboot.mod chain.mod multiboot.mod reboot.mod halt.mod \
vbe.mod vbetest.mod vbeinfo.mod video.mod gfxterm.mod \
videotest.mod play.mod bitmap.mod tga.mod cpuid.mod serial.mod ata.mod \
- vga.mod
+ vga.mod memdisk.mod
# For biosdisk.mod.
biosdisk_mod_SOURCES = disk/i386/pc/biosdisk.c
@@ -263,4 +263,9 @@ vga_mod_SOURCES = term/i386/pc/vga.c
vga_mod_CFLAGS = $(COMMON_CFLAGS)
vga_mod_LDFLAGS = $(COMMON_LDFLAGS)
+# For memdisk.mod.
+memdisk_mod_SOURCES = disk/memdisk.c
+memdisk_mod_CFLAGS = $(COMMON_CFLAGS)
+memdisk_mod_LDFLAGS = $(COMMON_LDFLAGS)
+
include $(srcdir)/conf/common.mk
diff -x '*.mk' -Nurp grub2/disk/memdisk.c grub2.memdisk/disk/memdisk.c
--- grub2/disk/memdisk.c 1970-01-01 01:00:00.0 +0100
+++ grub2.memdisk/disk/memdisk.c 2007-12-21 21:43:25.0 +0100
@@ -0,0 +1,92 @@
+/* memdisk.c - Access embedded memory disk. */
+/*
+ * GRUB -- GRand Unified Bootloader
+ * Copyright (C) 2007 Free Software Foundation, Inc.
+ *
+ * GRUB is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * GRUB is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with GRUB. If not, see http://www.gnu.org/licenses/.
+ */
+
+#include grub/disk.h
+#include grub/dl.h
+#include grub/machine/kernel.h
+#include grub/misc.h
+#include grub/mm.h
+#include grub/types.h
+
+static grub_addr_t memdisk_addr;
+
+static int
+grub_memdisk_iterate (int (*hook) (const char *name))
+{
+ return hook (memdisk);
+}
+
+static grub_err_t
+grub_memdisk_open (const char *name, grub_disk_t disk)
+{
+ if (grub_strcmp (name, memdisk))
+ return grub_error (GRUB_ERR_UNKNOWN_DEVICE, not a memdisk);
+
+ disk-total_sectors = grub_memdisk_image_size / GRUB_DISK_SECTOR_SIZE;
+ disk-id = (int) 'mdsk';
+
+ disk-has_partitions = 0;
+ disk-data = NULL;
+
+ return GRUB_ERR_NONE;
+}
+
+static void
+grub_memdisk_close (grub_disk_t disk __attribute((unused)))
+{
+}
+
+static grub_err_t
+grub_memdisk_read (grub_disk_t disk, grub_disk_addr_t sector,
+ grub_size_t size, char *buf)
+{
+ grub_memcpy (buf, memdisk_addr + (sector GRUB_DISK_SECTOR_BITS), size GRUB_DISK_SECTOR_BITS);
+ return 0;
+}
+
+static grub_err_t
+grub_memdisk_write (grub_disk_t disk, grub_disk_addr_t sector,
+ grub_size_t size,
Re: Creating grub/grub2/grldr.mbr bootrom with makerom
On Dec 22, 2007 3:04 AM, Robert Millan [EMAIL PROTECTED] wrote: How well does compression work for GRUB 2 ? core.img is already compressed (with lzo); if LZMA makes better results perhaps it'd be a good idea to switch. in fact, lzma is must better than lzo, a simple test with stage2 from grub 0.97: original: 99834 lzo : 70945 lzo -9 : 60546 lzma: 51402 For compressed image core.img: original: 65045 lzma: 60087 -- Bean ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
Re: embedding filesystems in core image (Re: moving ata initialisation to a command)
On Dec 22, 2007 4:54 AM, Robert Millan [EMAIL PROTECTED] wrote: On Fri, Dec 21, 2007 at 07:04:05PM +0100, Robert Millan wrote: I'd prefer to support embedding filesystems in core.img instead. This solves the problem for every situation instead of just PC boot media (e.g. LinuxBIOS ELF payload image). Agreed, I would like to see such feature. Also for stuff like diskless boot. What I had in mind was the following. We could perhaps make it possible to add all files using grub-mkimage, not only modules. When you add modules, those will be loaded automatically. All other files will be ignored. Another filesystem module has to be written. It scans over these files and will make them available. This can be simple and elegant. I do not like the idea of adding a diskimage or so. Are you interested in working on this? Well, I already have an almost-working implementation. I guess I'll have to defend it then! :-P Really, it's clean and elegant. Very simple and flexible. Once I get the address detection right, that is.. Please have a look at the attached patch. This allows adding an arbitrary number of files. No extra filesystem module is needed, since any of the supported filesystems can be used with this feature: genisoimage -o filesystem_image -r stuff grub-mkimage -o core.img -m filesystem_image Yes, i think this concept is great, and i just think of an improvement for the module. we can add support for a raw file system which store files compactly, so that very little space is wasted on file system structure, for example, this is a possible implementation: OFFSET_TO_NEXT_ITEM FILE1_NAME_LEN FILE1_NAME FILE1_DATA OFFSET_TO_NEXT_ITEM FILE2_NAME_LEN FILE2_NAME FILE2_DATA This is useful in situation where the size of core.img is limited, like bootrom. -- Bean ___ Grub-devel mailing list Grub-devel@gnu.org http://lists.gnu.org/mailman/listinfo/grub-devel
