On 28.08.2007 20:41, ron minnich wrote:
> Round 2.
>
> I had not realized that lar had changed. So all my patches were
> against the wrong files.
>
> This code is against the right files in lar. I have also done a fair
> amount of refactoring as I needed it.
>
> don't get upset about the close of the fd right after mmap; it's legal
> and even recommended.
>
> Comments welcome as usual.
>
> ron
> p.s. I can not get the LARCHIVE problem to NOT happen now. So, until
> we fix it right, let's take my wrong fix. Three of us have tried and
> failed to fix this, so, let's assume it's not trivial.
I have added LARCHIVE debugging code to your patch below to find out why
and where the problem happens. Could you post the log of your patch with
my added debugging? I hope we can see an obvious fix through that.
Other than that, this patch is Acked by me, but please wait for an
additional ack from somebody else.
Carl-Daniel
> ------------------------------------------------------------------------
>
> This patch is revised based on comments.
>
> It also includes a demo of how to do a PIC initram, and:
>
> EMERGENCY PATCH! Please see patch to lib/lar.c and include/lar.h for the
> MAGIC constant. This fixes a bug I hit just now.
>
> This patch also includes an EXPERT option for enabling no-ELF mode.
> The system will default to old behaviour. See Kconfig in the root.
>
> I still wish to kill ELF mode very soon, however.
>
> LAR is a very capable format. With two simple extensions, we can use
> LAR to replace all that we are using ELF for now. This change can
> really make life better:
> 1. we can use streaming decompress instead of the current "uncompress
> elf to memory and then copy segments" approach. So we can get rid of
> THIS hardcode:
> #define UNCOMPRESS_AREA (0x400000)
> 2. A simple lar l can show ALL segments, including payload segments
> 3. It's really easy to see where things will go in memory, and catch problems
> 4. We can figure out an ELF input file is bogus BEFORE we flash, not
> AFTER we flash and try to boot it
> 5. did I mention streaming decompress?
> 6. We no longer have to worry about where we decompress the elf in
> memory (this problem was causing trouble when the payload was a linux
> kernel -- it was so big)
> 7. Since we have a load address, we can create this lar entry:
> normal/cmdline
> and specify that it be loaded at a place where linux will find it as
> the cmdline.
> 8. The decision on whether to XIP can be made in the LAR entry, not in
> hardcode. For example, if initram needs to be XIP, set the load
> address to 0xffffffff. Done.
>
> The change is simple. Add a load address and entry point to the lar
> header. Extend the lar tool to parse the elf file and create multiple
> lar segments. It looks like this:
> normal/payload0 (33192 bytes, lzma compressed to 18088 bytes @0x38
> load @0x100000, entry 0x105258)
> normal/payload1 (72 bytes, lzma compressed to 47 bytes @0x4718 load
> @0x1225a0, entry 0x105258)
> normal/option_table (932 bytes @0x4798 load @0, entry 0)
> normal/stage2 (33308 bytes, lzma compressed to 15474 bytes @0x4b78
> load @0, entry 0)
> normal/initram (4208 bytes @0x8828 load @0, entry 0)
> linuxbios.bootblock (16384 bytes @0xfc000 load @0, entry 0)
>
> note that the payload is now payload/segment0, payload/segment1, etc. I've
> extended
> linuxbios to look for these. Note that you can now see all the things
> that get loaded ;they're no longer hidden in an ELF header somewhere.
> Elf failures are gone!
>
> Note that I've left legacy elf support in, for now, but recommend we
> get rid of it as soon as possible.
>
> patch attached. This is a first pass. lar.c needs some refactoring but
> I want to get the cmdline going. You can now have a linux payload and
> it will uncompress with no problems.
>
> This has been tested with filo and BOCHS.
>
> Signed-off-by: Ronald G. Minnich <[EMAIL PROTECTED]>
> Index: Kconfig
> ===================================================================
> --- Kconfig (revision 480)
> +++ Kconfig (working copy)
> @@ -53,6 +53,25 @@
> help
> Append an extra string to the end of the LinuxBIOS version.
>
> +config NOELF
> + bool "Don't use ELF for payloads"
> + depends EXPERT
> + default n
> + help
> + Until now, LinuxBIOS has used elf for the payload. There are many
> problems
> + this, not least being the inefficiency -- the ELF has to be
> decompressed to
> + memory and then the segments have to be copied. Plus, lar can't see
> the segments
> + in the elf -- to see all segments, you have to extract the elf and
> run readelf on it.
> + There are problems with collisions of the decompressed ELF location
> in memory
> + and the segment locations in memory.
> + Finally, validation of the ELF is done at run time, once you have
> flashed the
> + FLASH and rebooted the machine. Boot time is really not the time you
> want to find
> + out your ELF payload is broken.
> + With this option, LinuxBIOS will direct lar to break each elf segment
> into a LAR
> + entry. ELF will not be used at all. Note that (for now) LinuxBIOS is
> backward
> + compatible -- if you put an ELF payload in, LinuxBIOS can still parse
> it. We hope
> + to remove ELF entirely in the future.
> +
> config BEEPS
> bool "Enable beeps upon certain LinuxBIOS events"
> depends EXPERT
> Index: include/lar.h
> ===================================================================
> --- include/lar.h (revision 480)
> +++ include/lar.h (working copy)
> @@ -52,9 +52,10 @@
>
> #include <types.h>
>
> -#define MAGIC "LARCHIVE"
> +/* see note in lib/lar.c as to why this is ARCHIVE and not LARCHIVE */
> +#define MAGIC "ARCHIVE"
> #define MAX_PATHLEN 1024
> -
> +/* NOTE -- This and the user-mode lar.h are NOT IN SYNC. Be careful. */
> struct lar_header {
> char magic[8];
> u32 len;
> @@ -62,7 +63,14 @@
> u32 checksum;
> u32 compchecksum;
> u32 offset;
> + /* Compression:
> + * 0 = no compression
> + * 1 = lzma
> + * 2 = nrv2b
> + */
> u32 compression;
> + u32 entry; /* we might need to make this u64 */
> + u32 loadaddress; /* ditto */
> };
>
> struct mem_file {
> @@ -70,6 +78,8 @@
> int len;
> u32 reallen;
> u32 compression;
> + void *entry;
> + void *loadaddress;
> };
>
> /* Prototypes. */
> @@ -77,5 +87,6 @@
> int copy_file(struct mem_file *archive, char *filename, void *where);
> int run_file(struct mem_file *archive, char *filename, void *where);
> int execute_in_place(struct mem_file *archive, char *filename);
> -
> +int run_address(void *f);
> +void *load_file(struct mem_file *archive, char *filename);
> #endif /* LAR_H */
> Index: mainboard/emulation/qemu-x86/initram.c
> ===================================================================
> --- mainboard/emulation/qemu-x86/initram.c (revision 480)
> +++ mainboard/emulation/qemu-x86/initram.c (working copy)
> @@ -19,10 +19,17 @@
>
> #include <console.h>
>
> +
> +/* This is how we force an absolute call when we are in PIC code (which this
> file is)
> + * If there is a nicer way to do this, I'd like to hear it. Ideally, we
> could tell
> + * gcc to force abs jumps on certain symbols.
> + */
> +int (*pk)(int msg_level, const char *fmt, ...) = printk;
> +
> int main(void)
> {
> - printk(BIOS_INFO, "RAM init code started.\n");
> - printk(BIOS_INFO, "Nothing to do.\n");
> + pk(BIOS_INFO, "RAM init code started.\n");
> + pk(BIOS_INFO, "Nothing to do.\n");
>
> return 0;
> }
> Index: mainboard/emulation/qemu-x86/Makefile
> ===================================================================
> --- mainboard/emulation/qemu-x86/Makefile (revision 480)
> +++ mainboard/emulation/qemu-x86/Makefile (working copy)
> @@ -41,14 +41,17 @@
> #
>
> INITRAM_OBJ = $(obj)/mainboard/$(MAINBOARDDIR)/initram.o
> +$(obj)/mainboard/$(MAINBOARDDIR)/initram.o:
> $(src)/mainboard/$(MAINBOARDDIR)/initram.c
> + cc -c $(INITCFLAGS) -fPIE $(src)/mainboard/$(MAINBOARDDIR)/initram.c
> -o $(obj)/mainboard/$(MAINBOARDDIR)/initram.o
> +
> # These are possibly not permanent
> -INITRAM_OBJ += $(obj)/lib/console.o $(obj)/lib/vtxprintf.o
> $(obj)/lib/uart8250.o $(obj)/arch/x86/post_code.o
> +#INITRAM_OBJ += $(obj)/lib/console.o $(obj)/lib/vtxprintf.o
> $(obj)/lib/uart8250.o $(obj)/arch/x86/post_code.o
>
> $(obj)/linuxbios.initram: $(obj)/stage0.init $(obj)/stage0.o $(INITRAM_OBJ)
> $(Q)# initram links against stage0
> $(Q)printf " LD $(subst $(shell pwd)/,,$(@))\n"
> - $(Q)$(LD) -Ttext 0x80000 $(INITRAM_OBJ) \
> - --entry=main -o $(obj)/linuxbios.initram.o
> + $(Q)$(LD) $(INITRAM_OBJ) \
> + --entry=main -R $(obj)/stage0.o -o $(obj)/linuxbios.initram.o
> $(Q)printf " OBJCOPY $(subst $(shell pwd)/,,$(@))\n"
> $(Q)$(OBJCOPY) -O binary $(obj)/linuxbios.initram.o \
> $(obj)/linuxbios.initram
> Index: lib/lar.c
> ===================================================================
> --- lib/lar.c (revision 480)
> +++ lib/lar.c (working copy)
> @@ -31,6 +31,13 @@
> #define ntohl(x) (x)
> #endif
>
> +int run_address(void *f)
> +{
> + int (*v) (void);
> + v = f;
> + return v();
> +}
> +
> int find_file(struct mem_file *archive, char *filename, struct mem_file
> *result)
> {
> char *walk, *fullname;
> @@ -42,29 +49,103 @@
>
> for (walk = archive->start;
> (walk - 1) < (char *)(archive->start + archive->len - 1 ); walk +=
> 16) {
> - if (strcmp(walk, MAGIC) != 0)
> + /* I am leaving this code in here because it is so dangerous.
> MAGIC is
> + * #define'd to a string. That string lives in data space. All
> of the 1M linuxbios
> + * image is a LAR file. Therefore, this search can walk ALL of
> linuxbios.
> + * IF the MAGIC string (in code space) just happens to be
> 16-byte aligned,
> + * Then the strcmp will succeed, and you will match a non-LAR
> entry,
> + * and you are screwed. can this happen? YES!
> + * LAR: Attempting to open 'fallback/initram'.
> + * LAR: Start 0xfff00000 len 0x100000
> + * LAR: current filename is normal/payload
> + * LAR: current filename is normal/option_table
> + * LAR: current filename is normal/stage2
> + * LAR: current filename is normal/initram
> + * LAR: current filename is R: it matches %s @ %p
> + * That garbage is there because the pointer is in the middle
> of a bunch
> + * of non-null-terminated junk. The fix is easy, as you can
> see.
> + * if (strcmp(walk, MAGIC) != 0)
> + * continue;
> + * And, yes, this did indeed fix the problem!
> + */
> + if (walk[0] != 'L')
> continue;
>
> + if (strcmp(&walk[1], MAGIC) != 0)
> + continue;
> +
Maybe change the code above to add add extra debugging code:
> + printk(BIOS_SPEW, "LAR: looking for archive header at %p",
> walk);
> + if (walk[0] != 'L') {
> + printk(BIOS_SPEW, "LAR: no L magic at %p", walk);
> continue;
> + }
>
> + if (strcmp(&walk[1], MAGIC) != 0) {
> + printk(BIOS_SPEW, "LAR: no ARCHIVE magic at %p", walk);
> + continue;
> + }
> +
> + printk(BIOS_SPEW, "LAR: full magic at %p", walk);
> header = (struct lar_header *)walk;
> fullname = walk + sizeof(struct lar_header);
>
> - printk(BIOS_SPEW, "LAR: current filename is %s\n", fullname);
> + printk(BIOS_SPEW, "LAR: search for %s\n", fullname);
> // FIXME: check checksum
>
> if (strcmp(fullname, filename) == 0) {
> + printk(BIOS_SPEW, "LAR: FOUND %s @ %p\n", fullname,
> header);
> result->start = walk + ntohl(header->offset);
> result->len = ntohl(header->len);
> result->reallen = ntohl(header->reallen);
> result->compression = ntohl(header->compression);
> + result->entry = (void *)ntohl(header->entry);
> + result->loadaddress = (void
> *)ntohl(header->loadaddress);
> + printk(BIOS_SPEW,
> + "start %p len %d reallen %d compression %x entry %p
> loadaddress %p\n",
> + result->start, result->len, result->reallen,
> + result->compression, result->entry,
> result->loadaddress);
> return 0;
> }
> // skip file
> walk += (ntohl(header->len) + ntohl(header->offset) -
> 1) & 0xfffffff0;
> }
> + printk(BIOS_SPEW, "NO FILE FOUND\n");
> return 1;
> }
>
> +
> +void *load_file(struct mem_file *archive, char *filename)
> +{
> + int ret;
> + struct mem_file result;
> + void *where;
> + void *entry;
> +
> + ret = find_file(archive, filename, &result);
> + if (ret) {
> + printk(BIOS_INFO, "LAR: load_file: No such file '%s'\n",
> + filename);
> + return (void *)-1;
> + }
> + entry = result.entry;
> + where = result.loadaddress;
> + printk(BIOS_SPEW, "LAR: Compression algorithm #%i used\n",
> result.compression);
> + /* no compression */
> + if (result.compression == 0) {
> + memcpy(where, result.start, result.len);
> + return entry;
> + }
> +#ifdef CONFIG_COMPRESSION_LZMA
> + /* lzma */
> + unsigned long ulzma(unsigned char *src, unsigned char *dst);
> + if (result.compression == 1) {
> + ulzma(result.start, where);
> + return entry;
> + }
> +#endif
> +#ifdef CONFIG_COMPRESSION_NRV2B
> + /* nrv2b */
> + unsigned long unrv2b(u8 *src, u8 *dst, unsigned long *ilen_p);
> + if (result.compression == 2) {
> + int tmp;
> + unrv2b(result.start, where, &tmp);
> + return entry;
> + }
> +#endif
> + printk(BIOS_INFO, "LAR: Compression algorithm #%i not supported!\n",
> result.compression);
> + return (void *)-1;
> +}
> +
> +/* FIXME -- most of copy_file should be replaced by load_file */
> int copy_file(struct mem_file *archive, char *filename, void *where)
> {
> int ret;
> @@ -85,7 +166,7 @@
> }
> #ifdef CONFIG_COMPRESSION_LZMA
> /* lzma */
> - unsigned long ulzma(unsigned char * src, unsigned char * dst);
> + unsigned long ulzma(unsigned char *src, unsigned char *dst);
> if (result.compression == 1) {
> ulzma(result.start, where);
> return 0;
> @@ -93,7 +174,7 @@
> #endif
> #ifdef CONFIG_COMPRESSION_NRV2B
> /* nrv2b */
> - unsigned long unrv2b(u8 * src, u8 * dst, unsigned long *ilen_p);
> + unsigned long unrv2b(u8 *src, u8 *dst, unsigned long *ilen_p);
> if (result.compression == 2) {
> int tmp;
> unrv2b(result.start, where, &tmp);
> @@ -113,6 +194,7 @@
> {
> int (*v) (void);
> struct mem_file result;
> + int ret;
>
> if ((u32) where != 0xFFFFFFFF) {
> if (copy_file(archive, filename, where)) {
> @@ -130,9 +212,11 @@
> }
> where = result.start;
> }
> -
> + printk(BIOS_SPEW, "where is %p\n", where);
> v = where;
> - return v();
> + ret = v();
> + printk(BIOS_SPEW, "run_file returns with %d\n", ret);
> + return ret;
> }
>
> /**
> Index: lib/stage2.c
> ===================================================================
> --- lib/stage2.c (revision 480)
> +++ lib/stage2.c (working copy)
> @@ -99,5 +99,6 @@
> write_tables();
> show_all_devs();
>
> + printk(BIOS_SPEW, "STAGE2 NOW RETURNING\n");
> return 0;
> }
> Index: arch/x86/stage1.c
> ===================================================================
> --- arch/x86/stage1.c (revision 480)
> +++ arch/x86/stage1.c (working copy)
> @@ -22,12 +22,16 @@
> #include <io.h>
> #include <console.h>
> #include <lar.h>
> +#include <string.h>
> #include <tables.h>
> #include <lib.h>
> #include <mc146818rtc.h>
> #include <post_code.h>
>
> -#define UNCOMPRESS_AREA 0x60000
> +/* ah, well, what a mess! This is a hard code. FIX ME but how?
> + * By getting rid of ELF ...
> + */
> +#define UNCOMPRESS_AREA (0x400000)
>
> /* these prototypes should go into headers */
> void uart_init(void);
> @@ -48,6 +52,24 @@
> post_code(0xf2);
> }
>
> +
> +/* until we get rid of elf */
> +int legacy(struct mem_file *archive, char *name, void *where, struct
> lb_memory *mem)
> +{
> + int ret;
> + struct mem_file result;
> + int elfboot_mem(struct lb_memory *mem, void *where, int size);
> + ret = copy_file(archive, name, where);
> + if (ret) {
> + printk(BIOS_ERR, "'%s' found, but could not load it.\n", name);
> + }
> +
> + ret = elfboot_mem(mem, where, result.reallen);
> +
> + printk(BIOS_ERR, "elfboot_mem returns %d\n", ret);
> + return -1;
> +}
> +
> /*
> * This function is called from assembler code whith its argument on the
> * stack. Force the compiler to generate always correct code for this case.
> @@ -57,6 +79,8 @@
> int ret;
> struct mem_file archive, result;
> int elfboot_mem(struct lb_memory *mem, void *where, int size);
> + void *entry;
> + int i;
>
> /* we can't statically init this hack. */
> unsigned char faker[64];
> @@ -144,20 +168,32 @@
> printk(BIOS_DEBUG, "Stage2 code done.\n");
>
> ret = find_file(&archive, "normal/payload", &result);
> - if (ret) {
> - printk(BIOS_ERR, "No such file '%s'.\n", "normal/payload");
> - die("FATAL: No payload found.\n");
> + if (! ret)
> + legacy(&archive, "normal/payload", (void *)UNCOMPRESS_AREA,
> mem);
> +
> +
> + /* new style lar boot. Install all the files in memory.
> + * By convention we take the entry point from the first
> + * one. Look for a cmdline as well.
> + */
> + for(i = 0, entry = (void *)0; ;i++) {
> + char filename[64];
> + void *newentry;
> + sprintf(filename, "normal/payload/segment%d", i);
> + archive.len = *(u32 *)0xfffffff4;
> + archive.start =(void *)(0UL-archive.len);
> + newentry = load_file(&archive, filename);
> + printk("newentry is %p\n", newentry);
> + if (newentry == (void *)-1)
> + break;
> + if (! entry)
> + entry = newentry;
> }
> - ret = copy_file(&archive, "normal/payload", (void *)UNCOMPRESS_AREA);
> - if (ret) {
> - printk(BIOS_ERR, "'%s' found, but could not load it.\n",
> "normal/payload");
> - die("FATAL: No usable payload found.\n");
> - }
> + printk(BIOS_SPEW, "all loaded, entry %p\n", entry);
> + run_address(entry);
>
> - ret = elfboot_mem(mem, (void *)UNCOMPRESS_AREA, result.reallen);
> + die("FATAL: No usable payload found.\n");
>
> - printk(BIOS_ERR, "elfboot_mem returns %d\n", ret);
> -
> die ("FATAL: Last stage returned to LinuxBIOS.\n");
> }
>
> Index: arch/x86/Makefile
> ===================================================================
> --- arch/x86/Makefile (revision 480)
> +++ arch/x86/Makefile (working copy)
> @@ -22,7 +22,7 @@
> ifeq ($(CONFIG_ARCH_X86),y)
>
> INITCFLAGS := $(CFLAGS) -I$(src)/include/arch/x86 -I$(src)/include \
> - -I$(obj) -fno-builtin
> + -I$(obj) -fno-builtin
>
> SILENT := >/dev/null 2>&1
>
> @@ -78,7 +78,7 @@
> endif
> $(Q)printf " LAR $(subst $(shell pwd)/,,$(@))\n"
> $(Q)rm -f $(obj)/linuxbios.rom
> - $(Q)cd $(obj)/lar.tmp && ../util/lar/lar $(COMPRESSFLAG) -c \
> + $(Q)cd $(obj)/lar.tmp && ../util/lar/lar $(PARSEELF) $(COMPRESSFLAG) -c
> \
> ../linuxbios.rom \
> $(LARFILES) \
> -s $(ROM_SIZE) -b $(obj)/linuxbios.bootblock
> @@ -122,6 +122,11 @@
> endif
> endif
>
> +ifeq ($(CONFIG_NOELF), y)
> + PARSEELF = -e
> +else
> + PARSEELF =
> +endif
>
> STAGE0_OBJ := $(patsubst %,$(obj)/lib/%,$(STAGE0_LIB_OBJ)) \
> $(patsubst %,$(obj)/arch/x86/%,$(STAGE0_ARCH_X86_OBJ)) \
> @@ -143,6 +148,9 @@
> $(Q)printf " OBJCOPY $(subst $(shell pwd)/,,$(@))\n"
> $(Q)$(OBJCOPY) -O binary $(obj)/stage0.o $(obj)/stage0.init
>
> + $(Q)printf " OBJCOPY (stage0 link) $(subst $(shell pwd)/,,$(@))\n"
> + $(Q)$(OBJCOPY) --prefix-symbols=stage0 $(obj)/stage0.o
> $(obj)/stage0_link.o
> +
> $(Q)printf " TEST $(subst $(shell pwd)/,,$(@))\n"
> $(Q)test `wc -c < $(obj)/stage0.init` -gt 16128 && \
> printf "Error. Bootblock got too big.\n" || true
> Index: util/lar/lar.c
> ===================================================================
> --- util/lar/lar.c (revision 480)
> +++ util/lar/lar.c (working copy)
> @@ -30,13 +30,14 @@
> #include <sys/stat.h>
> #include <sys/mman.h>
>
> +#include "lar.h"
> #include "lib.h"
> -#include "lar.h"
>
> #define VERSION "v0.9.1"
> #define COPYRIGHT "Copyright (C) 2006-2007 coresystems GmbH"
>
> static int isverbose = 0;
> +static int iselfparse = 0;
> static long larsize = 0;
> static char *bootblock = NULL;
> enum compalgo algo = none;
> @@ -44,7 +45,7 @@
> static void usage(char *name)
> {
> printf("\nLAR - the LinuxBIOS Archiver " VERSION "\n" COPYRIGHT "\n\n"
> - "Usage: %s [-cxal] archive.lar [[[file1] file2] ...]\n\n", name);
> + "Usage: %s [-ecxal] archive.lar [[[file1] file2] ...]\n\n",
> name);
> printf("Examples:\n");
> printf(" lar -c -s 32k -b bootblock myrom.lar foo nocompress:bar\n");
> printf(" lar -a myrom.lar foo blob:baz\n");
> @@ -66,13 +67,18 @@
> printf(" \ta 'm' suffix to multiple the size by
> 1024*1024.\n");
> printf(" -b [bootblock]\tSpecify the bootblock blob\n");
> printf(" -C [lzma|nrv2b]\tSpecify the compression method to use\n\n");
> + printf(" -e pre-parse the payload ELF into LAR segments.
> Recommended\n\n");
>
> printf("General options\n");
> printf(" -v\tEnable verbose mode\n");
> printf(" -V\tShow the version\n");
> printf(" -h\tShow this help\n");
> printf("\n");
> +}
>
> +int elfparse(void)
> +{
> + return iselfparse;
> }
>
> /* Add a human touch to the LAR size by allowing suffixes:
> @@ -209,6 +215,7 @@
> {"list", 0, 0, 'l'},
> {"size", 1, 0, 's'},
> {"bootblock", 1, 0, 'b'},
> + {"elfparse", 1, 0, 'e'},
> {"verbose", 0, 0, 'v'},
> {"version", 0, 0, 'V'},
> {"help", 0, 0, 'h'},
> @@ -220,7 +227,7 @@
> exit(1);
> }
>
> - while ((opt = getopt_long(argc, argv, "acC:xls:b:vVh?",
> + while ((opt = getopt_long(argc, argv, "acC:xels:b:vVh?",
> long_options, &option_index)) != EOF) {
> switch (opt) {
> case 'a':
> @@ -240,6 +247,9 @@
> case 'l':
> larmode = LIST;
> break;
> + case 'e':
> + iselfparse = 1;
> + break;
> case 'x':
> larmode = EXTRACT;
> break;
> Index: util/lar/lar.h
> ===================================================================
> --- util/lar/lar.h (revision 480)
> +++ util/lar/lar.h (working copy)
> @@ -60,6 +60,7 @@
> typedef uint32_t u32;
> typedef uint8_t u8;
>
> +/* NOTE -- This and the linuxbios lar.h are NOT IN SYNC. Be careful. */
> struct lar_header {
> char magic[8];
> u32 len;
> @@ -73,6 +74,8 @@
> * 2 = nrv2b
> */
> u32 compression;
> + u32 entry; /* we might need to make this u64 */
> + u32 loadaddress; /* ditto */
> };
>
> /**\struct
> Index: util/lar/lib.h
> ===================================================================
> --- util/lar/lib.h (revision 480)
> +++ util/lar/lib.h (working copy)
> @@ -38,6 +38,7 @@
>
> /* prototypes for lar.c functions */
> int verbose(void);
> +int elfparse(void);
> long get_larsize(void);
> char *get_bootblock(void);
>
> @@ -50,13 +51,24 @@
> struct file *get_files(void);
> void free_files(void);
>
> +/* Prototypes for ELF functions */
> +int iself(char *filebuf);
> +
> +/* Prototypes for in-memory LAR operations */
> +int lar_process_name(char *name, char **pfilename, char **ppathname,
> + enum compalgo *thisalgo);
> +u32 lar_compress(char *ptr, ssize_t size, char *temp, enum compalgo
> *thisalgo);
> +int lar_add_entry(struct lar *lar, char *pathname, void *data,
> + u32 complen, u32 reallen, u32 loadaddress, u32 entry,
> + enum compalgo thisalgo);
> /* Prototypes for the LAR I/O functions */
> +char *mapfile(char *filename, u32 *size);
> struct lar * lar_new_archive(const char *archive, unsigned int size);
> struct lar * lar_open_archive(const char *archive);
> void lar_close_archive(struct lar *lar);
>
> void lar_list_files(struct lar *lar, struct file *files);
> -int lar_add_file(struct lar *lar, const char *name);
> +int lar_add_file(struct lar *lar, char *name);
> int lar_add_bootblock(struct lar *lar, const char *bootblock);
> int lar_extract_files(struct lar *lar, struct file *files);
>
> Index: util/lar/stream.c
> ===================================================================
> --- util/lar/stream.c (revision 480)
> +++ util/lar/stream.c (working copy)
> @@ -32,9 +32,10 @@
> #include <sys/mman.h>
> #include <netinet/in.h>
> #include <libgen.h>
> +#include <elf.h>
>
> +#include "lar.h"
> #include "lib.h"
> -#include "lar.h"
>
> /**
> * \def err(fmt,args...)
> @@ -44,7 +45,126 @@
>
> extern enum compalgo algo;
>
> +/* ELF processing */
> /**
> + * Given a ptr to data, determine if the data is an ELF image.
> + * @param filebuf pointer to the data
> + * @return 1 if ELF, 0 if not
> + */
> +int iself(char *filebuf)
> +{
> + Elf32_Ehdr *ehdr;
> + /* validate elf header */
> + ehdr = (Elf32_Ehdr *)filebuf;
> + if (memcmp(ehdr->e_ident, ELFMAG, 4))
> + return 0;
> + return 1;
> +}
> +
> +/**
> + * Output all the ELF segments for a given file
> + * @param lar The LAR Archoe
> + * @param name The LAR name
> + * @param filebuf The ELF file
> + * @param filelen Size of the ELF file
> + * @param algo The recommend compression algorithm
> + * Return 0 on success, -1 on failure
> + */
> +int output_elf_segments(struct lar *lar, char *name, char *filebuf,
> + int filelen, enum compalgo algo)
> +{
> + int ret;
> + Elf32_Phdr *phdr;
> + Elf32_Ehdr *ehdr;
> + u32 entry;
> + int i;
> + int size;
> + unsigned char *header;
> + char ename[64];
> + int headers;
> + char *temp;
> + enum compalgo thisalgo;
> + u32 complen;
> +
> + /* Allocate a temporary buffer to compress into - this is unavoidable,
> + because we need to make sure that the compressed data will fit in
> + the LAR, and we won't know the size of the compressed data until
> + we actually compress it */
> +
> + temp = calloc(filelen, 1);
> +
> + if (temp == NULL) {
> + err("Out of memory.\n");
> + return -1;
> + }
> +
> + /* validate elf header */
> + ehdr = (Elf32_Ehdr *)filebuf;
> + headers = ehdr->e_phnum;
> + header = (unsigned char *)ehdr;
> + if (verbose())
> + fprintf(stderr, "Type %d machine %d version %d entry %p phoff
> %d shoff %d flags %#x hsize %d phentsize %d phnum %d s_hentsize %d s_shnum %d
> \n",
> + ehdr->e_type,
> + ehdr->e_machine,
> + ehdr->e_version,
> + (void *)ehdr->e_entry,
> + ehdr->e_phoff,
> + ehdr->e_shoff,
> + ehdr->e_flags,
> + ehdr->e_ehsize,
> + ehdr->e_phentsize,
> + ehdr->e_phnum,
> + ehdr->e_shentsize,
> + ehdr->e_shnum);
> + phdr = (Elf32_Phdr *)&(header[ehdr->e_phoff]);
> +
> + if (verbose())
> + fprintf(stderr, "%s: header %p #headers %d\n", __FUNCTION__,
> ehdr, headers);
> + entry = ehdr->e_entry;
> + for(i = 0; i < headers; i++) {
> + /* Ignore data that I don't need to handle */
> + if (phdr[i].p_type != PT_LOAD) {
> + if (verbose())
> + fprintf(stderr, "Dropping non PT_LOAD
> segment\n");
> + continue;
> + }
> + if (phdr[i].p_memsz == 0) {
> + if (verbose())
> + fprintf(stderr, "Dropping empty segment\n");
> + continue;
> + }
> + thisalgo = algo;
> + if (verbose())
> + fprintf(stderr, "New segment addr 0x%ulx size 0x%ulx
> offset 0x%ulx filesize 0x%ulx\n",
> + (u32)phdr[i].p_paddr, (u32)phdr[i].p_memsz,
> + (u32)phdr[i].p_offset, (u32)phdr[i].p_filesz);
> + /* Clean up the values */
> + size = phdr[i].p_filesz;
> + if (phdr[i].p_filesz > phdr[i].p_memsz) {
> + size = phdr[i].p_memsz;
> + }
> + if (verbose()) {
> + fprintf(stderr, "(cleaned up) New segment addr %p size
> 0x%#x offset 0x%x\n",
> + (void *)phdr[i].p_paddr, size,
> phdr[i].p_offset);
> + fprintf(stderr, "Copy to %p from %p for %d bytes\n",
> + (unsigned char *)phdr[i].p_paddr,
> + &header[phdr[i].p_offset], size);
> + fprintf(stderr, "entry %ux loadaddr %ux\n",
> + entry, phdr[i].p_paddr);
> + }
> + /* ok, copy it out */
> + sprintf(ename, "%s/segment%d", name, i);
> + complen = lar_compress(filebuf, size, temp, &thisalgo);
> + ret = lar_add_entry(lar, ename, &header[phdr[i].p_offset],
> + complen, size,
> + phdr[i].p_paddr, entry, thisalgo);
> + }
> + return 0;
> +out:
> + return -1;
> +}
> +
> +/**
> * Given a size, return the offset of the bootblock (including the
> * header)
> * @param size Size of the LAR archive
> @@ -259,7 +379,7 @@
> }
>
> return lar;
> - err:
> +err:
> lar_close_archive(lar);
>
> /* Don't leave a halfbaked LAR laying around */
> @@ -315,7 +435,7 @@
>
> return lar;
>
> - err:
> +err:
> lar_close_archive(lar);
> return NULL;
> }
> @@ -535,110 +655,139 @@
> }
>
> /**
> - * Add a new file to the LAR archive
> - * @param lar The LAR archive to write into
> - * @param name The name of the file to add
> - * @return 0 on success, or -1 on failure
> + * Given a pathname in the form [option;]path, determine the file name,
> + * LAR path name, and compression algorithm.
> + * @param name name in the [option:][./]path form
> + * @param pfilename reference pointer to file name -- this is modified
> + * @param ppathname reference pointer to LAR path name -- this is modified
> + * @param thisalgo pointer to algorithm, which can be modified by path name
> + * @return 0 success, or -1 on failure (i.e. a bad name)
> */
> -int lar_add_file(struct lar *lar, const char *name)
> +int lar_process_name(char *name, char **pfilename, char **ppathname,
> + enum compalgo *thisalgo)
> {
> - char *filename, *ptr, *temp;
> - char *pathname;
> + char *filename = name, *pathname = name;
>
> - enum compalgo thisalgo;
> - struct lar_header *header;
> - u32 offset;
> - int ret, fd, hlen;
> - u32 complen;
> - int pathlen;
> - struct stat s;
> - u32 *walk, csum;
> -
> - /* Find the beginning of the available space in the LAR */
> - offset = lar_empty_offset(lar);
> -
> - thisalgo = algo;
> -
> - filename = (char *) name;
> -
> if (!strncmp(name, "nocompress:",11)) {
> filename += 11;
> - thisalgo = none;
> + *thisalgo = none;
> }
>
> + /* this is dangerous */
> if (filename[0] == '.' && filename[1] == '/')
> filename += 2;
>
> pathname = strchr(filename, ':');
>
> if (pathname != NULL) {
> - *pathname = '\0';
> - pathname++;
> + *pathname = '\0';
> + pathname++;
>
> - if (!strlen(pathname)) {
> - err("Invalid pathname specified.\n");
> - return -1;
> - }
> + if (!strlen(pathname)) {
> + err("Invalid pathname specified.\n");
> + return -1;
> + }
> }
> else
> pathname = filename;
> + *pfilename = filename;
> + *ppathname = pathname;
> + return 0;
> +}
>
> +
> +/**
> + * Given a pathname, open and mmap the file.
> + * @param filename
> + * @param size pointer to returned size
> + * @return ptr to mmap'ed area on success, or MAP_FAILED on failure
> + */
> +char *mapfile(char *filename, u32 *size)
> +{
> + int fd;
> + struct stat s;
> + char *ptr;
> +
> /* Open the file */
> fd = open(filename, O_RDONLY);
>
> if (fd == -1) {
> err("Unable to open %s\n", filename);
> - return -1;
> + return MAP_FAILED;
> }
>
> if (fstat(fd, &s)) {
> err("Unable to stat the file %s\n", filename);
> close(fd);
> - return -1;
> + return MAP_FAILED;
> }
>
> - /* Allocate a temporary buffer to compress into - this is unavoidable,
> - because we need to make sure that the compressed data will fit in
> - the LAR, and we won't know the size of the compressed data until
> - we actually compress it */
> -
> - temp = calloc(s.st_size, 1);
> -
> - if (temp == NULL) {
> - err("Out of memory.\n");
> - return -1;
> - }
> -
> ptr = mmap(0, s.st_size, PROT_READ, MAP_SHARED, fd, 0);
> + close(fd);
>
> if (ptr == MAP_FAILED) {
> err("Unable to map %s\n", filename);
> - close(fd);
> - free(temp);
> - return -1;
> + return MAP_FAILED;
> }
> + *size = s.st_size;
> + return ptr;
> +}
>
> +/**
> + * Compress an area according to an algorithm. If the area grows,
> + * use no compression.
> + * @param ptr data to be compressed
> + * @param size size of the data
> + * @param temp destination of compressed data
> + * @param thisalgo pointer to algorithm -- this can be modified
> + * @return size of compressed data
> + */
> +u32 lar_compress(char *ptr, ssize_t size, char *temp, enum compalgo
> *thisalgo)
> +{
> + u32 complen;
> + compress_functions[*thisalgo](ptr, size, temp, &complen);
>
> - /* Do the compression step */
> - compress_functions[thisalgo](ptr, s.st_size, temp, &complen);
> -
> - if (complen >= s.st_size && (thisalgo != none)) {
> - thisalgo = none;
> - compress_functions[thisalgo](ptr, s.st_size, temp, &complen);
> + if (complen >= size && (*thisalgo != none)) {
> + *thisalgo = none;
> + compress_functions[*thisalgo](ptr, size, temp, &complen);
> }
> + return complen;
> +}
>
> - munmap(ptr, s.st_size);
> - close(fd);
> +/**
> + * Add a new entry to the LAR archive
> + * @param lar The LAR archive to write into
> + * @param pathname The name of the segment
> + * @param data The data for the segment
> + * @param complen The compressed length of the segment
> + * @param reallen The real (uncompressed) length of the segment
> + * @param loadaddress The load address of the segment
> + * @param entry The entry point of the segment
> + * @param thisalgo The compression algorithm
> + * @return 0 on success, or -1 on failure
> + */
> +int lar_add_entry(struct lar *lar, char *pathname, void *data,
> + u32 complen, u32 reallen, u32 loadaddress, u32 entry,
> + enum compalgo thisalgo)
> +{
> + struct lar_header *header;
> + int ret, hlen;
> + int pathlen;
> + u32 *walk, csum;
> + u32 offset;
>
> - pathlen = strlen(pathname) + 1 > MAX_PATHLEN ? MAX_PATHLEN :
> strlen(pathname) + 1;
> + /* Find the beginning of the available space in the LAR */
> + offset = lar_empty_offset(lar);
>
> + pathlen = strlen(pathname) + 1 > MAX_PATHLEN ?
> + MAX_PATHLEN : strlen(pathname) + 1;
> +
> /* Figure out how big our header will be */
> hlen = sizeof(struct lar_header) + pathlen;
> hlen = (hlen + 15) & 0xFFFFFFF0;
>
> if (offset + hlen + complen >= get_bootblock_offset(lar->size)) {
> err("Not enough room in the LAR to add the file.\n");
> - free(temp);
> return -1;
> }
>
> @@ -651,16 +800,17 @@
>
> memcpy(header, MAGIC, 8);
> header->compression = htonl(thisalgo);
> - header->reallen = htonl(s.st_size);
> + header->reallen = htonl(reallen);
> header->len = htonl(complen);
> header->offset = htonl(hlen);
> -
> + header->loadaddress = htonl(loadaddress);
> + header->entry = htonl(entry);
> /* Copy the path name */
> strncpy((char *) (lar->map + offset + sizeof(struct lar_header)),
> pathname, pathlen - 1);
>
> /* Copy in the data */
> - memcpy(lar->map + (offset + hlen), temp, complen);
> + memcpy(lar->map + (offset + hlen), data, complen);
>
> /* Figure out the checksum */
>
> @@ -671,7 +821,58 @@
> csum += ntohl(*walk);
> }
> header->checksum = htonl(csum);
> + return 0;
> +}
>
> +/**
> + * Add a new file to the LAR archive
> + * @param lar The LAR archive to write into
> + * @param name The name of the file to add
> + * @return 0 on success, or -1 on failure
> + */
> +int lar_add_file(struct lar *lar, char *name)
> +{
> + char *filename, *ptr, *temp;
> + char *pathname;
> +
> + enum compalgo thisalgo;
> + struct lar_header *header;
> + int ret, hlen;
> + u32 complen;
> + int pathlen;
> + u32 size;
> +
> + thisalgo = algo;
> + lar_process_name(name, &filename, &pathname, &thisalgo);
> +
> + ptr = mapfile(filename, &size);
> +
> + if (elfparse() && iself(ptr)) {
> + output_elf_segments(lar, pathname, ptr, size, thisalgo);
> + return 0;
> + }
> +
> + /* This is legacy stuff. */
> +
> + /* Allocate a temporary buffer to compress into - this is unavoidable,
> + because we need to make sure that the compressed data will fit in
> + the LAR, and we won't know the size of the compressed data until
> + we actually compress it */
> +
> + temp = calloc(size, 1);
> +
> + if (temp == NULL) {
> + err("Out of memory.\n");
> + munmap(ptr, size);
> + return -1;
> + }
> +
> + complen = lar_compress(ptr, size, temp, &thisalgo);
> +
> + munmap(ptr, size);
> +
> + ret = lar_add_entry(lar, pathname, temp, complen, size, 0, 0, thisalgo);
> +
> free(temp);
> - return 0;
> + return ret;
> }
> Index: util/lar/Makefile
> ===================================================================
> --- util/lar/Makefile (revision 480)
> +++ util/lar/Makefile (working copy)
> @@ -17,7 +17,6 @@
> ## along with this program; if not, write to the Free Software
> ## Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA, 02110-1301 USA
> ##
> -
> LAROBJ := lar.o stream.o lib.o
>
> LARDIR := lardir
> @@ -29,7 +28,6 @@
> LARDIR += nrv2bdir
>
> LAROBJ_ABS := $(patsubst %,$(obj)/util/lar/%,$(LAROBJ))
> -
> lardir:
> $(Q)printf " BUILD LAR\n"
> $(Q)mkdir -p $(obj)/util/lar
>
--
http://www.hailfinger.org/
--
linuxbios mailing list
[email protected]
http://www.linuxbios.org/mailman/listinfo/linuxbios
