Re: [Libhugetlbfs-devel] [PATCH] tools: hugeedit: Modify binaries to set default remapping behavior V3
On Wed, Jul 23, 2008 at 07:34:27PM +, Adam Litke wrote:
> The recently added relinking method makes it possible to choose from multiple
> ways to remap segments at run-time via the HUGTLB_ELFMAP environment variable.
> If nothing is specified in that variable, then no remapping will occur.
> Sometimes it is desirable to set a default remapping mode so that
> HUGETLB_ELFMAP does not need to be set. This semantic applied to the original
> remapping algorithm.
>
> This patch adds a utility that can modify the PF_LINUX_HUGETLB segment flags
> in
> the ELF haeder of a binary to control which segments will be remapped by
> default. The tool will also display this information.
>
> Changes since V2 (Thanks again Mel)
> - Support for remapping more of the ELF header if one page is not enough
> - is_text() checks that p_filesz == p_memsz in addition to permissions
> - Rename scan_phdrs* to update_phdrs*
> - Print segment start and end addresses
> - Whitespace
>
> Changes since V1 (Thanks Mel for the review):
> - New --text, --data, and --disable options
> - Binary default modes are always displayed
> - Binary is opened for writing only when updates have been requested
>
> Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
> ---
>
> Makefile |2 -
> hugeedit.c | 219
>
> 2 files changed, 220 insertions(+), 1 deletions(-)
> create mode 100644 hugeedit.c
>
>
> diff --git a/Makefile b/Makefile
> index 6c0132a..4c58948 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -3,7 +3,7 @@ EXEDIR = /bin
>
> LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o
> INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
> -INSTALL_BIN = hugectl
> +INSTALL_BIN = hugectl hugeedit
> INSTALL_HEADERS = hugetlbfs.h
> LDSCRIPT_TYPES = B BDT
> LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64
> diff --git a/hugeedit.c b/hugeedit.c
> new file mode 100644
> index 000..a74722d
> --- /dev/null
> +++ b/hugeedit.c
> @@ -0,0 +1,219 @@
> +/*
> + * libhugetlbfs - Easy use of Linux hugepages
> + * Copyright (C) 2008 Adam Litke, IBM Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public License
> + * as published by the Free Software Foundation; either version 2.1 of
> + * the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#define _GNU_SOURCE
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include "libhugetlbfs_internal.h"
> +
> +#define PF_LINUX_HUGETLB 0x10
> +extern int optind;
> +extern char *optarg;
> +
> +/*
> + * All MAP_* options are tagged with MAP_BASE to differentiate them as
> options
> + * in the options parser. This must be removed before they are compared.
> + */
> +#define MAP_BASE 0x1000
> +#define MAP_DISABLE 0x0001
> +#define MAP_TEXT 0x0002
> +#define MAP_DATA 0x0004
> +
> +void print_usage()
> +{
> + printf("hugeedit [options] target\n");
> + printf("options:\n");
> + printf("--text- Remap program text into huge pages by default\n");
> + printf("--data- Remap program data into huge pages by default\n");
> + printf("--disable - Remap no segments into huge pages by default\n");
> + printf("--help- Print this usage information\n");
> +}
> +
> +int check_elf_wordsize(void *ehdr)
> +{
> + char *e_ident = (char *) ehdr;
> +
> + if (strncmp(e_ident, "\177ELF", 4)) {
> + printf("Not a valid ELF executable\n");
Although you do only have error levels in here I think I would use the same
form of error labeling we have elsewhere, ie. convert these all to ERROR().
However I would say its not worth bringing all the levels baggage over here.
Just maintain the form then if later we need the control we do not need to
convert them all again. Something like:
#define ERROR(format, ...)\
fprintf(stderr, "hugeedit: ERROR: " format, ##__VA_ARGS__)
ERROR("Not a valid ELF executable\n");
> + exit(EXIT_FAILURE);
> + }
> +
> + switch (e_ident[EI_CLASS]) {
> + case ELFCLASS32:
> + case ELFCLASS64:
> + return e_ident[EI_CLASS];
> + default:
> + printf("Can't determine word size\n");
> +
Re: [Libhugetlbfs-devel] [PATCH 1/2] build: Cleanup tools build
On Fri, Jul 18, 2008 at 06:39:58PM +, Adam Litke wrote: > To properly support building and installing tools (hugectl, hugeedit, hugecfg, > etc), some changes to the initial method of building hugectl are required. > This patch creates wildcard rules that can be used for building either 32 or > 64 > bit versions of all tools. Additionally, a native wordsize is selected for > each architecture so that only one binary (of the desired default word size) > is > built and installed. > > Signed-off-by: Adam Litke <[EMAIL PROTECTED]> > --- > > Makefile | 25 + > 1 files changed, 17 insertions(+), 8 deletions(-) > > > diff --git a/Makefile b/Makefile > index 28b7ec1..608d36b 100644 > --- a/Makefile > +++ b/Makefile > @@ -2,9 +2,8 @@ PREFIX = /usr/local > EXEDIR = /bin > > LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o > -OBJS = hugectl.o > INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a > -INSTALL_OBJ = hugectl > +INSTALL_BIN = hugectl > INSTALL_HEADERS = hugetlbfs.h > LDSCRIPT_TYPES = B BDT > LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64 > @@ -28,6 +27,7 @@ CC64 = gcc -m64 > ELF64 = elf64ppc > LIB64 = lib64 > LIB32 = lib > +BIN_OBJ_DIR = obj64 > ifneq ($(BUILDTYPE),NATIVEONLY) > CC32 = gcc > ELF32 = elf32ppclinux > @@ -37,17 +37,20 @@ ifeq ($(ARCH),ppc) > CC32 = gcc > ELF32 = elf32ppclinux > LIB32 = lib > +BIN_OBJ_DIR = obj32 > else > ifeq ($(ARCH),i386) > CC32 = gcc > ELF32 = elf_i386 > LIB32 = lib > +BIN_OBJ_DIR = obj32 > else > ifeq ($(ARCH),x86_64) > CC64 = gcc -m64 > ELF64 = elf_x86_64 > LIB64 = lib64 > LIB32 = lib > +BIN_OBJ_DIR = obj64 > ifneq ($(BUILDTYPE),NATIVEONLY) > CC32 = gcc -m32 > ELF32 = elf_i386 > @@ -56,11 +59,13 @@ else > ifeq ($(ARCH),ia64) > CC64 = gcc > LIB64 = lib64 > +BIN_OBJ_DIR = obj64 > CFLAGS += -DNO_ELFLINK > else > ifeq ($(ARCH),sparc64) > CC64 = gcc -m64 > LIB64 = lib64 > +BIN_OBJ_DIR = obj64 > CFLAGS += -DNO_ELFLINK > else > $(error "Unrecognized architecture ($(ARCH))") > @@ -146,7 +151,7 @@ tests:tests/all > tests/%: > $(MAKE) -C tests $* > > -tools: $(INSTALL_OBJ) > +tools: $(foreach file,$(INSTALL_BIN),$(BIN_OBJ_DIR)/$(file)) > > check: all > cd tests; ./run_tests.sh > @@ -237,15 +242,18 @@ $(OBJS):hugectl.c > @$(VECHO) CPP $@ > $(CC) $(CFLAGS) $(LIBPATHS) -o $@ -c $< > > -$(INSTALL_OBJ): $(OBJS) > - @$(VECHO) CC $@ > - $(CC) $(CFLAGS) -o $@ $< > +$(INSTALL_BIN:%=obj32/%): obj32/%: %.c > + @$(VECHO) CC32 $@ > + $(CC32) $(CPPFLAGS) $(CFLAGS) -o $@ $^ > + > +$(INSTALL_BIN:%=obj64/%): obj64/%: %.c > + @$(VECHO) CC64 $@ > + $(CC64) $(CPPFLAGS) $(CFLAGS) -o $@ $^ I am a little confused by why we consider bit size at all for the 'host' applications. We seem to build one or other only for host applications even where the host supports both sizes. But the makefile has to specify which they are. Could we not just have 'objnative' (perhaps just 'obj' in line with obj32/64) for native host applications, and just always compile them without any sort of -mXX, with $(CC). Clearly the applications are compilable as either size as we do that for different architectures? Something like this: BIN_OBJ_DIR='obj' tools: $(foreach file,$(INSTALL_BIN),$(BIN_OBJ_DIR)/$(file)) $(INSTALL_BIN:%=$(BIN_OBJ_DIR)/%): $(BIN_OBJ_DIR)/%: %.c @$(VECHO) CCNATIVE $@ $(CC) $(CPPFLAGS) $(CFLAGS) -o $@ $^ > > clean: > @$(VECHO) CLEAN > rm -f *~ *.o *.so *.a *.d *.i core a.out $(VERSION) > rm -rf obj* > - rm -f $(OBJ) $(INSTALL_OBJ) > rm -f ldscripts/*~ > rm -f libhugetlbfs-sock > $(MAKE) -C tests clean > @@ -282,7 +290,8 @@ install: libs tools $(OBJDIRS:%=%/install) > $(INSTALL_OBJSCRIPT:%=objscript.%) > $(INSTALL) -m 644 $(INSTALL_LDSCRIPTS:%=ldscripts/%) > $(DESTDIR)$(LDSCRIPTDIR) > $(INSTALL) -d $(DESTDIR)$(BINDIR) > $(INSTALL) -d $(DESTDIR)$(EXEDIR) > - $(INSTALL) -m 755 $(INSTALL_OBJ) $(DESTDIR)$(EXEDIR) > + for x in $(INSTALL_BIN); do \ > + $(INSTALL) -m 755 $(BIN_OBJ_DIR)/$$x $(DESTDIR)$(EXEDIR); done > for x in $(INSTALL_OBJSCRIPT); do \ > $(INSTALL) -m 755 objscript.$$x $(DESTDIR)$(BINDIR)/$$x; done > cd $(DESTDIR)$(BINDIR) && ln -sf ld.hugetlbfs ld -apw - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH] tools: hugeedit: Modify binaries to set default remapping behavior V3
On Thu, Jul 24, 2008 at 09:50:30AM +0100, Andy Whitcroft wrote:
> > + switch (ret) {
> > + case '?':
> > + case 'h':
> > + print_usage();
> > + exit(EXIT_SUCCESS);
> > +
Having made the opposite error in hugectl I think that this is probabally
wrong for the '?' case. As Mel pointed out to me if you ask for the help
'-h' then its done what you asked successfully. BUT if you get an option
wrong '-Z' then its actually going to return '?' and that has not done
what you wanted and is an error. S. Probabally this should be
case '?':
print_usage();
exit(EXIT_FAILURE);
case 'h':
print_usage();
exit(EXIT_SUCCESS);
-apw
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 5/9] hugectl: add remap support via --text, --data, --bss, and --disable
On Thu, Jul 24, 2008 at 10:42:11AM +1000, David Gibson wrote: > On Wed, Jul 23, 2008 at 04:33:40PM +0100, Andy Whitcroft wrote: > > Add control over remap via the --text, --data, --bss, and --disable > > options. The first three request mapping of those segments, the last > > disables all remap. Where the combinations requested cannot be exactly > > handled then the request is "widened" to get that segment remapped, > > for example if you request --data or --bss in isolation then both are > > remapped and a warning emitted. > > Here's one idea I thought about when the new remapping technique was > introduced. We could add a secondary way of specifying what to remap > in elflink.c where the segments to remapped are explicitly listed (by > program header index, presumably). hugectl could in theory use more > information than just the segment permissions to deduce which segments > are the data/bss/etc, then use this mode to specify this to the > library. > > I doubt it's actually all that useful in practice, but it's an option > to bear in mind if we do find the read-only/read-write distinction > insufficient. It is funny you should say that right now as I was thinking about this myself. I think we can handle that in the same variable, and essentially in the same format. If we were to define the current format as something like: "a comma separated list of segments to remap, where unambigious the comma is optional" Then the current valid options of (left) are trivially detectable degenerate forms of (right): R R W W RW R,W ie. if we see RW, assume R,W. Then we can also later support per segment mapping using their segment numbers: 0,2,5 I also strongly agree that at this point we should consider it, and make sure any change to this format does not preclude simple introduction of segment number based mapping. -apw - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [RFC] Fallback to base pages when hugepages are unavailable by specifying GHP_FALLBACK to get_huge_pages()
get_huge_pages() is an API for the allocation of hugepage-backed regions of
memory. In the event it returns NULL, the application is expected to use
mmap() or malloc() if base pages would be suitable. This is boiler-plate
code that is an unnecessary burden on the application developer. This patch
adds GHP_FALLBACK to indicate get_huge_pages() should use base pages if
hugepages are not available.
Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
---
alloc.c| 25 -
hugetlbfs.h|1 +
tests/get_huge_pages.c | 36
tests/run_tests.sh |2 +-
4 files changed, 62 insertions(+), 2 deletions(-)
diff --git a/alloc.c b/alloc.c
index 65bd063..2a95fb2 100644
--- a/alloc.c
+++ b/alloc.c
@@ -29,6 +29,23 @@
#include "hugetlbfs.h"
#include "libhugetlbfs_internal.h"
+/* Allocate base pages if huge page allocation fails */
+static void *fallback_base_pages(size_t len, ghp_t flags)
+{
+ void *buf;
+ DEBUG("Insufficient free huge pages, falling back to base pages\n");
+ buf = mmap(NULL, len,
+ PROT_READ|PROT_WRITE,
+ MAP_PRIVATE|MAP_ANONYMOUS,
+ 0, 0);
+ if (buf == MAP_FAILED) {
+ WARNING("Base page fallback failed: %s\n", strerror(errno));
+ buf = NULL;
+ }
+
+ return buf;
+}
+
/**
* get_huge_pages - Allocate an amount of memory backed by huge pages
* len: Size of the region to allocate
@@ -58,7 +75,13 @@ void *get_huge_pages(size_t len, ghp_t flags)
MAP_PRIVATE, heap_fd, len);
if (buf == MAP_FAILED) {
close(heap_fd);
- WARNING("New heap segment map failed: %s\n", strerror(errno));
+
+ /* Try falling back to base pages if allowed */
+ if (flags & GHP_FALLBACK)
+ return fallback_base_pages(len, flags);
+
+ WARNING("New heap segment map (flags: 0x%lX) failed: %s\n",
+ flags, strerror(errno));
return NULL;
}
diff --git a/hugetlbfs.h b/hugetlbfs.h
index 3945836..30d1fa1 100644
--- a/hugetlbfs.h
+++ b/hugetlbfs.h
@@ -40,6 +40,7 @@ long dump_proc_pid_maps(void);
* by the current implementation of the library
*/
typedef unsigned long ghp_t;
+#define GHP_FALLBACK (0x01UL)/* Use base pages if necessary */
#define GHP_DEFAULT(0)
/* Direct alloc functions */
diff --git a/tests/get_huge_pages.c b/tests/get_huge_pages.c
index b05e849..cc19a73 100644
--- a/tests/get_huge_pages.c
+++ b/tests/get_huge_pages.c
@@ -45,12 +45,48 @@ void test_get_huge_pages(int num_hugepages)
FAIL("hugepage was not correctly freed");
}
+void test_GHP_FALLBACK(void)
+{
+ int err;
+ long hpage_size = check_hugepagesize();
+ long num_hugepages = read_meminfo("HugePages_Total:");
+
+ /* We should be able to allocate the whole pool */
+ void *p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
+ if (p == NULL)
+ FAIL("test_GHP_FALLBACK(GHP_DEFAULT) failed for %ld hugepages",
+ num_hugepages);
+ memset(p, 1, hpage_size);
+ err = test_addr_huge(p + (num_hugepages -1) * hpage_size);
+ if (err != 1)
+ FAIL("Returned page is not hugepage");
+ free_huge_pages(p);
+
+ /* We should fail allocating too much */
+ num_hugepages++;
+ p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
+ if (p != NULL)
+ FAIL("test_GHP_FALLBACK() for %ld expected fail, got success",
num_hugepages);
+
+ /* GHP_FALLBACK should succeed by allocating base pages */
+ p = get_huge_pages(num_hugepages * hpage_size, GHP_FALLBACK);
+ if (p == NULL)
+ FAIL("test_GHP_FALLBACK(GHP_FALLBACK) failed for %ld hugepages",
+ num_hugepages);
+ memset(p, 1, hpage_size);
+ err = test_addr_huge(p + (num_hugepages -1) * hpage_size);
+ if (err == 1)
+ FAIL("Returned page is not a base page");
+ free_huge_pages(p);
+}
+
int main(int argc, char *argv[])
{
test_init(argc, argv);
check_free_huge_pages(4);
test_get_huge_pages(1);
test_get_huge_pages(4);
+ test_GHP_FALLBACK();
PASS();
}
diff --git a/tests/run_tests.sh b/tests/run_tests.sh
index c98179d..7ef41f1 100755
--- a/tests/run_tests.sh
+++ b/tests/run_tests.sh
@@ -273,7 +273,7 @@ check_linkhuge_tests
run_test truncate_sigbus_versus_oom
# Test direct allocation API
-run_test get_huge_pages
+run_test HUGETLB_VERBOSE=1 get_huge_pages
# Test hugetlbfs filesystem quota accounting
run_test quota
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Mo
Re: [Libhugetlbfs-devel] [PATCH 8/9] hugectl: add support for requesting which library to use
On Wed, Jul 23, 2008 at 04:51:06PM +0100, Mel Gorman wrote:
> On (23/07/08 16:33), Andy Whitcroft didst pronounce:
> > Add support for requesting a specific library set for preload. This adds
> > the --library option which takes an optional argument. Without an argument
> > it requests use of the specific libraries installed with the version
> > of hugectl in use. An argument is treated as a prefix to the library
> > directories and the approprate 32/64 bit library directories are added.
> >
> > Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
> > ---
> > Makefile |9 +++-
> > hugectl.c | 64
> > +
> > 2 files changed, 72 insertions(+), 1 deletions(-)
> >
> > diff --git a/Makefile b/Makefile
> > index d69a365..6571672 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -101,6 +101,13 @@ BINDIR = $(PREFIX)/share/libhugetlbfs
> > EXEDIR = $(PREFIX)/bin
> > DOCDIR = $(PREFIX)/share/doc/libhugetlbfs
> >
> > +ifdef LIB32
> > +LIBPATHS += -DLIB32='"$(LIB32)"' -DLIBDIR32='"$(LIBDIR32)"'
> > +endif
> > +ifdef LIB64
> > +LIBPATHS += -DLIB64='"$(LIB64)"' -DLIBDIR64='"$(LIBDIR64)"'
> > +endif
> > +
> > EXTRA_DIST = \
> > README \
> > HOWTO \
> > @@ -228,7 +235,7 @@ obj64/%.s: %.c
> >
> > $(OBJS): hugectl.c
> > @$(VECHO) CPP $@
> > - $(CC) $(CFLAGS) -o $@ -c $<
> > + $(CC) $(CFLAGS) $(LIBPATHS) -o $@ -c $<
> >
> > $(INSTALL_OBJ):$(OBJS)
> > @$(VECHO) CC $@
> > diff --git a/hugectl.c b/hugectl.c
> > index 9513bf1..adffa45 100644
> > --- a/hugectl.c
> > +++ b/hugectl.c
> > @@ -31,6 +31,7 @@
> > #include
> > #include
> > #include
> > +#include
> >
> > #define _GNU_SOURCE /* for getopt_long */
> > #include
> > @@ -61,6 +62,8 @@ void print_usage()
> > fprintf(stderr, " (malloc
> > space)\n");
> > fprintf(stderr, " --no-preload, Disable
> > preloading the libhugetlbfs\n");
> > fprintf(stderr, " library\n");
> > + fprintf(stderr, " --no-library, Suppress the
> > default library\n");
> > + fprintf(stderr, " --library , Select a
> > library prefix for preload\n");
> > }
>
> This --no-library thing is new and the explanation doesn't tell me what
> is going on. Does it mean, "use the first library found in the system
> path instead of the default" ?
There are three options really:
1) use the library path that matches this hugectl
2) use the library path I am specifying
3) do nothing about the library path at all
You sensibly wanted no options to mean (1). But we then need an option for
(3). We want an option which means "ok just leave the library selection
alone".
> --system-library Use the system library instead of the default
> (Default: path to lib here)
>
> ?
Its not the system library necessarily its the library that would be
found with the current settings, which might be the system default or
might be one the user has specified via LD_LIBRARY_PATH. So 'system' is
probabally wrong. 'default' would cover it, but would be confusing wrt
the "Default:" above too. Hmmm. Names are hard:
--library-use-path
--library-use-system-path
--library-system-path
??
>
> >
> > int verbose_level = VERBOSITY_DEFAULT;
> > @@ -92,6 +95,9 @@ void setup_environment(char *var, char *val)
> >
> > #define LONG_NO_PRELOAD(LONG_BASE | 'p')
> >
> > +#define LONG_NO_LIBRARY(LONG_BASE | 'L')
> > +#define LONG_LIBRARY (LONG_BASE | 'l')
> > +
> > /*
> > * Mapping selectors, one bit per remappable/backable area as requested
> > * by the user. These are also used as returns from getopts where they
> > @@ -135,6 +141,49 @@ void setup_mappings(int which)
> > setup_environment("HUGETLB_MORECORE", "yes");
> > }
> >
> > +#define LIBRARY_DISABLE ((void *)-1)
> > +
> > +void library_path(char *path)
> > +{
> > + char val[NAME_MAX] = "";
> > + char *env;
> > +
> > + /*
> > +* Select which libraries we wish to use. No argument implies
> > +* use the libraries included with this version of hugectl.
> > +* Else it is assumed to be a prefix to the 32/64 bit directories
> > +* both of which are added, where available.
> > +*/
> > + env = getenv("LD_LIBRARY_PATH");
> > + if (!path) {
> > + snprintf(val, sizeof(val), ""
> > +#ifdef LIBDIR32
> > + LIBDIR32 ":"
> > +#endif
> > +#ifdef LIBDIR64
> > + LIBDIR64 ":"
> > +#endif
> > + "%s", env ? env : "");
> > + } else {
> > + snprintf(val, sizeof(val), ""
> > +#ifdef LIBDIR32
> > + "%s/" LIB32 ":"
> > +#endif
> > +#ifdef LIBDIR64
> > + "%s/" LIB64 ":"
> > +#endif
> > + "%s",
> > +#ifdef LIBDIR32
> > + path,
> > +#endif
> > +#ifdef LIBDIR64
> > + path,
> > +#endif
> > + env ? env : "");
[Libhugetlbfs-devel] [PATCH 0/3] [V1] hugectl cleanups II
Following this email are three further cleanups for the basic hugectl infrastructure, and its interface to the library debugging. These have come out of the ongoing work on hugectl, but stand alone as cleanups. Following this email are three patches: debug: expose default and maximum verbosity level -- which exposes the library default and maxmimum verbosity levels so that other applications can track the same levels, hugectl: convert exit status to POSIX standard codes -- which fixes up the exit statuses to be POSIX compliant, and hugectl: simplify per-option help text -- which simplifies specification of the help text for new options. These should be reasonably obvious and clean, please consider for application. -apw Andy Whitcroft (3): debug: expose default and maximum verbosity level hugectl: convert exit status to POSIX standard codes hugectl: simplify per-option help text debug.c |2 +- hugectl.c| 16 libhugetlbfs_debug.h |3 +++ 3 files changed, 16 insertions(+), 5 deletions(-) - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [RFC] Fallback to base pages when hugepages are unavailable by specifying GHP_FALLBACK to get_huge_pages()
On Thu, Jul 24, 2008 at 11:37:17AM +0100, Mel Gorman wrote:
> get_huge_pages() is an API for the allocation of hugepage-backed regions of
> memory. In the event it returns NULL, the application is expected to use
> mmap() or malloc() if base pages would be suitable. This is boiler-plate
> code that is an unnecessary burden on the application developer. This patch
> adds GHP_FALLBACK to indicate get_huge_pages() should use base pages if
Ok, does this mean fallback to base pages or could it mean fallback to
the biggest size you have? I mean this more from an interface point of
view rather than this implementation. That is should there be like:
GHP_FALLBACK_BASE
GHP_FALLBACK_NOBIGGERTHAN
GHP_FALLBACK_ANY
And GHP_FALLBACK -> _NOBIGGERTHAN (yes the names suck). This implementation
would be appropriate for any of these, as its falling back to the biggest it
supports, base pages.
> hugepages are not available.
>
> Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
> ---
> alloc.c| 25 -
> hugetlbfs.h|1 +
> tests/get_huge_pages.c | 36
> tests/run_tests.sh |2 +-
> 4 files changed, 62 insertions(+), 2 deletions(-)
>
> diff --git a/alloc.c b/alloc.c
> index 65bd063..2a95fb2 100644
> --- a/alloc.c
> +++ b/alloc.c
> @@ -29,6 +29,23 @@
> #include "hugetlbfs.h"
> #include "libhugetlbfs_internal.h"
>
> +/* Allocate base pages if huge page allocation fails */
> +static void *fallback_base_pages(size_t len, ghp_t flags)
> +{
> + void *buf;
> + DEBUG("Insufficient free huge pages, falling back to base pages\n");
> + buf = mmap(NULL, len,
> + PROT_READ|PROT_WRITE,
> + MAP_PRIVATE|MAP_ANONYMOUS,
> + 0, 0);
> + if (buf == MAP_FAILED) {
> + WARNING("Base page fallback failed: %s\n", strerror(errno));
> + buf = NULL;
> + }
> +
> + return buf;
> +}
> +
> /**
> * get_huge_pages - Allocate an amount of memory backed by huge pages
> * len: Size of the region to allocate
> @@ -58,7 +75,13 @@ void *get_huge_pages(size_t len, ghp_t flags)
>MAP_PRIVATE, heap_fd, len);
> if (buf == MAP_FAILED) {
> close(heap_fd);
> - WARNING("New heap segment map failed: %s\n", strerror(errno));
> +
> + /* Try falling back to base pages if allowed */
> + if (flags & GHP_FALLBACK)
> + return fallback_base_pages(len, flags);
> +
> + WARNING("New heap segment map (flags: 0x%lX) failed: %s\n",
> + flags, strerror(errno));
> return NULL;
> }
>
> diff --git a/hugetlbfs.h b/hugetlbfs.h
> index 3945836..30d1fa1 100644
> --- a/hugetlbfs.h
> +++ b/hugetlbfs.h
> @@ -40,6 +40,7 @@ long dump_proc_pid_maps(void);
> * by the current implementation of the library
> */
> typedef unsigned long ghp_t;
> +#define GHP_FALLBACK (0x01UL)/* Use base pages if necessary */
> #define GHP_DEFAULT (0)
>
> /* Direct alloc functions */
> diff --git a/tests/get_huge_pages.c b/tests/get_huge_pages.c
> index b05e849..cc19a73 100644
> --- a/tests/get_huge_pages.c
> +++ b/tests/get_huge_pages.c
> @@ -45,12 +45,48 @@ void test_get_huge_pages(int num_hugepages)
> FAIL("hugepage was not correctly freed");
> }
>
> +void test_GHP_FALLBACK(void)
> +{
> + int err;
> + long hpage_size = check_hugepagesize();
> + long num_hugepages = read_meminfo("HugePages_Total:");
> +
> + /* We should be able to allocate the whole pool */
> + void *p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
> + if (p == NULL)
> + FAIL("test_GHP_FALLBACK(GHP_DEFAULT) failed for %ld hugepages",
> + num_hugepages);
> + memset(p, 1, hpage_size);
> + err = test_addr_huge(p + (num_hugepages -1) * hpage_size);
> + if (err != 1)
> + FAIL("Returned page is not hugepage");
> + free_huge_pages(p);
> +
> + /* We should fail allocating too much */
> + num_hugepages++;
> + p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
> + if (p != NULL)
> + FAIL("test_GHP_FALLBACK() for %ld expected fail, got success",
> num_hugepages);
> +
> + /* GHP_FALLBACK should succeed by allocating base pages */
> + p = get_huge_pages(num_hugepages * hpage_size, GHP_FALLBACK);
> + if (p == NULL)
> + FAIL("test_GHP_FALLBACK(GHP_FALLBACK) failed for %ld hugepages",
> + num_hugepages);
> + memset(p, 1, hpage_size);
> + err = test_addr_huge(p + (num_hugepages -1) * hpage_size);
> + if (err == 1)
> + FAIL("Returned page is not a base page");
> + free_huge_pages(p);
> +}
> +
> int main(int argc, char *argv[])
> {
> test_init(argc, argv);
> check_free_huge_pages(4);
> test_get_huge_pages(1);
> test_g
[Libhugetlbfs-devel] [PATCH 3/3] hugectl: simplify per-option help text
Simplify specification of per-option help text.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
---
hugectl.c |6 +-
1 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/hugectl.c b/hugectl.c
index d9b7040..8ba794c 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -40,11 +40,15 @@ extern int errno;
extern int optind;
extern char *optarg;
+#define OPTION(opts, text) fprintf(stderr, " %-25s %s\n", opts, text)
+#define CONT(text) fprintf(stderr, " %-25s %s\n", "", text)
+
void print_usage()
{
fprintf(stderr, "hugectl [options] target\n");
fprintf(stderr, "options:\n");
- fprintf(stderr, " --help, -h Prints this
message.\n");
+
+ OPTION("--help, -h", "Prints this message");
}
int main(int argc, char** argv)
--
1.5.6.GIT
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 2/3] hugectl: convert exit status to POSIX standard codes
Move to using the POSIX standard codes EXIT_FAILURE/EXIT_SUCCESS instead
of explicit numbers.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
---
hugectl.c | 10 +++---
1 files changed, 7 insertions(+), 3 deletions(-)
diff --git a/hugectl.c b/hugectl.c
index 10e8806..d9b7040 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -60,11 +60,15 @@ int main(int argc, char** argv)
ret = getopt_long(argc, argv, opts, long_opts, &index);
switch (ret) {
case '?':
+ print_usage();
+ exit(EXIT_FAILURE);
+
case 'h':
print_usage();
- exit(1);
+ exit(EXIT_SUCCESS);
default:
+ fprintf(stderr, "unparsed option %08x\n", ret);
ret = -1;
break;
}
@@ -73,10 +77,10 @@ int main(int argc, char** argv)
if ((argc - index) < 1) {
print_usage();
- exit(1);
+ exit(EXIT_FAILURE);
}
execvp(argv[index], &argv[index]);
fprintf(stderr, "Error calling execvp: '%s'\n", strerror(errno));
- exit(1);
+ exit(EXIT_FAILURE);
}
--
1.5.6.GIT
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [RFC] Fallback to base pages when hugepages are unavailable by specifying GHP_FALLBACK to get_huge_pages()
On (24/07/08 15:34), Andy Whitcroft didst pronounce:
> On Thu, Jul 24, 2008 at 11:37:17AM +0100, Mel Gorman wrote:
> > get_huge_pages() is an API for the allocation of hugepage-backed regions of
> > memory. In the event it returns NULL, the application is expected to use
> > mmap() or malloc() if base pages would be suitable. This is boiler-plate
> > code that is an unnecessary burden on the application developer. This patch
> > adds GHP_FALLBACK to indicate get_huge_pages() should use base pages if
>
> Ok, does this mean fallback to base pages or could it mean fallback to
> the biggest size you have?
It could be implemented to be either and the name could be fine. It's
just saying "give me the default hugepage size if you can and if not,
fallback to something else" which could be either smaller pages or a
base page.
> I mean this more from an interface point of
> view rather than this implementation. That is should there be like:
>
> GHP_FALLBACK_BASE
> GHP_FALLBACK_NOBIGGERTHAN
> GHP_FALLBACK_ANY
>
> And GHP_FALLBACK -> _NOBIGGERTHAN (yes the names suck). This implementation
> would be appropriate for any of these, as its falling back to the biggest it
> supports, base pages.
>
That seems excessively complicated to me. I'd prefer the developer not
have to be so specific and instead just get something sensible from the
interface without having to be too specific about it.
> > hugepages are not available.
> >
> > Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
> > ---
> > alloc.c| 25 -
> > hugetlbfs.h|1 +
> > tests/get_huge_pages.c | 36
> > tests/run_tests.sh |2 +-
> > 4 files changed, 62 insertions(+), 2 deletions(-)
> >
> > diff --git a/alloc.c b/alloc.c
> > index 65bd063..2a95fb2 100644
> > --- a/alloc.c
> > +++ b/alloc.c
> > @@ -29,6 +29,23 @@
> > #include "hugetlbfs.h"
> > #include "libhugetlbfs_internal.h"
> >
> > +/* Allocate base pages if huge page allocation fails */
> > +static void *fallback_base_pages(size_t len, ghp_t flags)
> > +{
> > + void *buf;
> > + DEBUG("Insufficient free huge pages, falling back to base pages\n");
> > + buf = mmap(NULL, len,
> > + PROT_READ|PROT_WRITE,
> > + MAP_PRIVATE|MAP_ANONYMOUS,
> > + 0, 0);
> > + if (buf == MAP_FAILED) {
> > + WARNING("Base page fallback failed: %s\n", strerror(errno));
> > + buf = NULL;
> > + }
> > +
> > + return buf;
> > +}
> > +
> > /**
> > * get_huge_pages - Allocate an amount of memory backed by huge pages
> > * len: Size of the region to allocate
> > @@ -58,7 +75,13 @@ void *get_huge_pages(size_t len, ghp_t flags)
> > MAP_PRIVATE, heap_fd, len);
> > if (buf == MAP_FAILED) {
> > close(heap_fd);
> > - WARNING("New heap segment map failed: %s\n", strerror(errno));
> > +
> > + /* Try falling back to base pages if allowed */
> > + if (flags & GHP_FALLBACK)
> > + return fallback_base_pages(len, flags);
> > +
> > + WARNING("New heap segment map (flags: 0x%lX) failed: %s\n",
> > + flags, strerror(errno));
> > return NULL;
> > }
> >
> > diff --git a/hugetlbfs.h b/hugetlbfs.h
> > index 3945836..30d1fa1 100644
> > --- a/hugetlbfs.h
> > +++ b/hugetlbfs.h
> > @@ -40,6 +40,7 @@ long dump_proc_pid_maps(void);
> > * by the current implementation of the library
> > */
> > typedef unsigned long ghp_t;
> > +#define GHP_FALLBACK (0x01UL)/* Use base pages if necessary
> > */
> > #define GHP_DEFAULT(0)
> >
> > /* Direct alloc functions */
> > diff --git a/tests/get_huge_pages.c b/tests/get_huge_pages.c
> > index b05e849..cc19a73 100644
> > --- a/tests/get_huge_pages.c
> > +++ b/tests/get_huge_pages.c
> > @@ -45,12 +45,48 @@ void test_get_huge_pages(int num_hugepages)
> > FAIL("hugepage was not correctly freed");
> > }
> >
> > +void test_GHP_FALLBACK(void)
> > +{
> > + int err;
> > + long hpage_size = check_hugepagesize();
> > + long num_hugepages = read_meminfo("HugePages_Total:");
> > +
> > + /* We should be able to allocate the whole pool */
> > + void *p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
> > + if (p == NULL)
> > + FAIL("test_GHP_FALLBACK(GHP_DEFAULT) failed for %ld hugepages",
> > + num_hugepages);
> > + memset(p, 1, hpage_size);
> > + err = test_addr_huge(p + (num_hugepages -1) * hpage_size);
> > + if (err != 1)
> > + FAIL("Returned page is not hugepage");
> > + free_huge_pages(p);
> > +
> > + /* We should fail allocating too much */
> > + num_hugepages++;
> > + p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
> > + if (p != NULL)
> > + FAIL("test_GHP_FALLBACK() for %ld expected fail, got success",
> > num_hugepages);
> > +
> > + /
Re: [Libhugetlbfs-devel] [RFC] Fallback to base pages when hugepages are unavailable by specifying GHP_FALLBACK to get_huge_pages()
On Thu, Jul 24, 2008 at 04:27:07PM +0100, Mel Gorman wrote:
> On (24/07/08 15:34), Andy Whitcroft didst pronounce:
> > On Thu, Jul 24, 2008 at 11:37:17AM +0100, Mel Gorman wrote:
> > > get_huge_pages() is an API for the allocation of hugepage-backed regions
> > > of
> > > memory. In the event it returns NULL, the application is expected to use
> > > mmap() or malloc() if base pages would be suitable. This is boiler-plate
> > > code that is an unnecessary burden on the application developer. This
> > > patch
> > > adds GHP_FALLBACK to indicate get_huge_pages() should use base pages if
> >
> > Ok, does this mean fallback to base pages or could it mean fallback to
> > the biggest size you have?
>
> It could be implemented to be either and the name could be fine. It's
> just saying "give me the default hugepage size if you can and if not,
> fallback to something else" which could be either smaller pages or a
> base page.
>
> > I mean this more from an interface point of
> > view rather than this implementation. That is should there be like:
> >
> > GHP_FALLBACK_BASE
> > GHP_FALLBACK_NOBIGGERTHAN
> > GHP_FALLBACK_ANY
> >
> > And GHP_FALLBACK -> _NOBIGGERTHAN (yes the names suck). This implementation
> > would be appropriate for any of these, as its falling back to the biggest it
> > supports, base pages.
> >
>
> That seems excessively complicated to me. I'd prefer the developer not
> have to be so specific and instead just get something sensible from the
> interface without having to be too specific about it.
Right, and I guess my point was probabally that we should have
GHP_FALLBACK mean something general, so that we can make it use any size
we want thats smaller than the size they asked for including small
pages. If the definition is that we have most flexibilty.
> > > hugepages are not available.
> > >
> > > Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
> > > ---
> > > alloc.c| 25 -
> > > hugetlbfs.h|1 +
> > > tests/get_huge_pages.c | 36
> > > tests/run_tests.sh |2 +-
> > > 4 files changed, 62 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/alloc.c b/alloc.c
> > > index 65bd063..2a95fb2 100644
> > > --- a/alloc.c
> > > +++ b/alloc.c
> > > @@ -29,6 +29,23 @@
> > > #include "hugetlbfs.h"
> > > #include "libhugetlbfs_internal.h"
> > >
> > > +/* Allocate base pages if huge page allocation fails */
> > > +static void *fallback_base_pages(size_t len, ghp_t flags)
> > > +{
> > > + void *buf;
> > > + DEBUG("Insufficient free huge pages, falling back to base pages\n");
> > > + buf = mmap(NULL, len,
> > > + PROT_READ|PROT_WRITE,
> > > + MAP_PRIVATE|MAP_ANONYMOUS,
> > > + 0, 0);
> > > + if (buf == MAP_FAILED) {
> > > + WARNING("Base page fallback failed: %s\n", strerror(errno));
> > > + buf = NULL;
> > > + }
> > > +
> > > + return buf;
> > > +}
> > > +
> > > /**
> > > * get_huge_pages - Allocate an amount of memory backed by huge pages
> > > * len: Size of the region to allocate
> > > @@ -58,7 +75,13 @@ void *get_huge_pages(size_t len, ghp_t flags)
> > >MAP_PRIVATE, heap_fd, len);
> > > if (buf == MAP_FAILED) {
> > > close(heap_fd);
> > > - WARNING("New heap segment map failed: %s\n", strerror(errno));
> > > +
> > > + /* Try falling back to base pages if allowed */
> > > + if (flags & GHP_FALLBACK)
> > > + return fallback_base_pages(len, flags);
> > > +
> > > + WARNING("New heap segment map (flags: 0x%lX) failed: %s\n",
> > > + flags, strerror(errno));
> > > return NULL;
> > > }
> > >
> > > diff --git a/hugetlbfs.h b/hugetlbfs.h
> > > index 3945836..30d1fa1 100644
> > > --- a/hugetlbfs.h
> > > +++ b/hugetlbfs.h
> > > @@ -40,6 +40,7 @@ long dump_proc_pid_maps(void);
> > > * by the current implementation of the library
> > > */
> > > typedef unsigned long ghp_t;
> > > +#define GHP_FALLBACK (0x01UL)/* Use base pages if necessary
> > > */
> > > #define GHP_DEFAULT (0)
> > >
> > > /* Direct alloc functions */
> > > diff --git a/tests/get_huge_pages.c b/tests/get_huge_pages.c
> > > index b05e849..cc19a73 100644
> > > --- a/tests/get_huge_pages.c
> > > +++ b/tests/get_huge_pages.c
> > > @@ -45,12 +45,48 @@ void test_get_huge_pages(int num_hugepages)
> > > FAIL("hugepage was not correctly freed");
> > > }
> > >
> > > +void test_GHP_FALLBACK(void)
> > > +{
> > > + int err;
> > > + long hpage_size = check_hugepagesize();
> > > + long num_hugepages = read_meminfo("HugePages_Total:");
> > > +
> > > + /* We should be able to allocate the whole pool */
> > > + void *p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
> > > + if (p == NULL)
> > > + FAIL("test_GHP_FALLBACK(GHP_DEFAULT) failed for %ld hugepages",
> > > + num_hugepages
Re: [Libhugetlbfs-devel] [PATCH 8/9] hugectl: add support for requesting which library to use
On (24/07/08 12:26), Andy Whitcroft didst pronounce:
> On Wed, Jul 23, 2008 at 04:51:06PM +0100, Mel Gorman wrote:
> > On (23/07/08 16:33), Andy Whitcroft didst pronounce:
> > > Add support for requesting a specific library set for preload. This adds
> > > the --library option which takes an optional argument. Without an
> > > argument
> > > it requests use of the specific libraries installed with the version
> > > of hugectl in use. An argument is treated as a prefix to the library
> > > directories and the approprate 32/64 bit library directories are added.
> > >
> > > Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
> > > ---
> > > Makefile |9 +++-
> > > hugectl.c | 64
> > > +
> > > 2 files changed, 72 insertions(+), 1 deletions(-)
> > >
> > > diff --git a/Makefile b/Makefile
> > > index d69a365..6571672 100644
> > > --- a/Makefile
> > > +++ b/Makefile
> > > @@ -101,6 +101,13 @@ BINDIR = $(PREFIX)/share/libhugetlbfs
> > > EXEDIR = $(PREFIX)/bin
> > > DOCDIR = $(PREFIX)/share/doc/libhugetlbfs
> > >
> > > +ifdef LIB32
> > > +LIBPATHS += -DLIB32='"$(LIB32)"' -DLIBDIR32='"$(LIBDIR32)"'
> > > +endif
> > > +ifdef LIB64
> > > +LIBPATHS += -DLIB64='"$(LIB64)"' -DLIBDIR64='"$(LIBDIR64)"'
> > > +endif
> > > +
> > > EXTRA_DIST = \
> > > README \
> > > HOWTO \
> > > @@ -228,7 +235,7 @@ obj64/%.s:%.c
> > >
> > > $(OBJS): hugectl.c
> > > @$(VECHO) CPP $@
> > > - $(CC) $(CFLAGS) -o $@ -c $<
> > > + $(CC) $(CFLAGS) $(LIBPATHS) -o $@ -c $<
> > >
> > > $(INSTALL_OBJ): $(OBJS)
> > > @$(VECHO) CC $@
> > > diff --git a/hugectl.c b/hugectl.c
> > > index 9513bf1..adffa45 100644
> > > --- a/hugectl.c
> > > +++ b/hugectl.c
> > > @@ -31,6 +31,7 @@
> > > #include
> > > #include
> > > #include
> > > +#include
> > >
> > > #define _GNU_SOURCE /* for getopt_long */
> > > #include
> > > @@ -61,6 +62,8 @@ void print_usage()
> > > fprintf(stderr, " (malloc
> > > space)\n");
> > > fprintf(stderr, " --no-preload, Disable
> > > preloading the libhugetlbfs\n");
> > > fprintf(stderr, " library\n");
> > > + fprintf(stderr, " --no-library, Suppress the
> > > default library\n");
> > > + fprintf(stderr, " --library , Select a
> > > library prefix for preload\n");
> > > }
> >
> > This --no-library thing is new and the explanation doesn't tell me what
> > is going on. Does it mean, "use the first library found in the system
> > path instead of the default" ?
>
> There are three options really:
>
> 1) use the library path that matches this hugectl
> 2) use the library path I am specifying
> 3) do nothing about the library path at all
>
> You sensibly wanted no options to mean (1). But we then need an option for
> (3). We want an option which means "ok just leave the library selection
> alone".
>
> > --system-libraryUse the system library instead of the default
> > (Default: path to lib here)
> >
> > ?
>
> Its not the system library necessarily its the library that would be
> found with the current settings, which might be the system default or
> might be one the user has specified via LD_LIBRARY_PATH. So 'system' is
> probabally wrong. 'default' would cover it, but would be confusing wrt
> the "Default:" above too. Hmmm. Names are hard:
>
> --library-use-path
> --library-use-system-path
> --library-system-path
>
--use-ld-library-path
Does Exactly What It Says On The Tin
> ??
>
> >
> > >
> > > int verbose_level = VERBOSITY_DEFAULT;
> > > @@ -92,6 +95,9 @@ void setup_environment(char *var, char *val)
> > >
> > > #define LONG_NO_PRELOAD (LONG_BASE | 'p')
> > >
> > > +#define LONG_NO_LIBRARY (LONG_BASE | 'L')
> > > +#define LONG_LIBRARY (LONG_BASE | 'l')
> > > +
> > > /*
> > > * Mapping selectors, one bit per remappable/backable area as requested
> > > * by the user. These are also used as returns from getopts where they
> > > @@ -135,6 +141,49 @@ void setup_mappings(int which)
> > > setup_environment("HUGETLB_MORECORE", "yes");
> > > }
> > >
> > > +#define LIBRARY_DISABLE ((void *)-1)
> > > +
> > > +void library_path(char *path)
> > > +{
> > > + char val[NAME_MAX] = "";
> > > + char *env;
> > > +
> > > + /*
> > > + * Select which libraries we wish to use. No argument implies
> > > + * use the libraries included with this version of hugectl.
> > > + * Else it is assumed to be a prefix to the 32/64 bit directories
> > > + * both of which are added, where available.
> > > + */
> > > + env = getenv("LD_LIBRARY_PATH");
> > > + if (!path) {
> > > + snprintf(val, sizeof(val), ""
> > > +#ifdef LIBDIR32
> > > + LIBDIR32 ":"
> > > +#endif
> > > +#ifdef LIBDIR64
> > > + LIBDIR64 ":"
> > > +#endif
> > > + "%s", env ? env : "");
> > > + } else {
> > > +
[Libhugetlbfs-devel] [PATCH 1/3] debug: expose default and maximum verbosity level
Expose the default and maximum verbosity levels via with the debugging level macros so that applications can use it to enable 'all' tracing. Use these in the main library. Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]> Acked-by: Mel Gorman <[EMAIL PROTECTED]> --- debug.c |2 +- libhugetlbfs_debug.h |3 +++ 2 files changed, 4 insertions(+), 1 deletions(-) diff --git a/debug.c b/debug.c index a2174f8..25a2843 100644 --- a/debug.c +++ b/debug.c @@ -27,7 +27,7 @@ #include "libhugetlbfs_internal.h" -int __hugetlbfs_verbose = 1; +int __hugetlbfs_verbose = VERBOSITY_DEFAULT; int __hugetlbfs_debug = 0; int __hugetlbfs_prefault = 1; char __hugetlbfs_hostname[64]; diff --git a/libhugetlbfs_debug.h b/libhugetlbfs_debug.h index 4ca26f3..7eb8f4e 100644 --- a/libhugetlbfs_debug.h +++ b/libhugetlbfs_debug.h @@ -27,4 +27,7 @@ #define DEBUG(...) REPORT(3, "DEBUG", ##__VA_ARGS__) #define DEBUG_CONT(...)REPORT_CONT(3, "DEBUG", ##__VA_ARGS__) +#define VERBOSITY_MAX 3 +#define VERBOSITY_DEFAULT 1 + #endif -- 1.5.6.GIT - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [RFC] Fallback to base pages when hugepages are unavailable by specifying GHP_FALLBACK to get_huge_pages()
On (24/07/08 16:35), Andy Whitcroft didst pronounce:
> On Thu, Jul 24, 2008 at 04:27:07PM +0100, Mel Gorman wrote:
> > On (24/07/08 15:34), Andy Whitcroft didst pronounce:
> > > On Thu, Jul 24, 2008 at 11:37:17AM +0100, Mel Gorman wrote:
> > > > get_huge_pages() is an API for the allocation of hugepage-backed
> > > > regions of
> > > > memory. In the event it returns NULL, the application is expected to use
> > > > mmap() or malloc() if base pages would be suitable. This is boiler-plate
> > > > code that is an unnecessary burden on the application developer. This
> > > > patch
> > > > adds GHP_FALLBACK to indicate get_huge_pages() should use base pages if
> > >
> > > Ok, does this mean fallback to base pages or could it mean fallback to
> > > the biggest size you have?
> >
> > It could be implemented to be either and the name could be fine. It's
> > just saying "give me the default hugepage size if you can and if not,
> > fallback to something else" which could be either smaller pages or a
> > base page.
> >
> > > I mean this more from an interface point of
> > > view rather than this implementation. That is should there be like:
> > >
> > > GHP_FALLBACK_BASE
> > > GHP_FALLBACK_NOBIGGERTHAN
> > > GHP_FALLBACK_ANY
> > >
> > > And GHP_FALLBACK -> _NOBIGGERTHAN (yes the names suck). This
> > > implementation
> > > would be appropriate for any of these, as its falling back to the biggest
> > > it
> > > supports, base pages.
> > >
> >
> > That seems excessively complicated to me. I'd prefer the developer not
> > have to be so specific and instead just get something sensible from the
> > interface without having to be too specific about it.
>
>
> Right, and I guess my point was probabally that we should have
> GHP_FALLBACK mean something general, so that we can make it use any size
> we want thats smaller than the size they asked for including small
> pages. If the definition is that we have most flexibilty.
>
Ah, so it's a matter of definition. I'll update the description
GHP_FALLBACK /* Use the default hugepage size if possible but fallback
* to smaller page sizes if necessary
*/
?
> > > > hugepages are not available.
> > > >
> > > > Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
> > > > ---
> > > > alloc.c| 25 -
> > > > hugetlbfs.h|1 +
> > > > tests/get_huge_pages.c | 36
> > > > tests/run_tests.sh |2 +-
> > > > 4 files changed, 62 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/alloc.c b/alloc.c
> > > > index 65bd063..2a95fb2 100644
> > > > --- a/alloc.c
> > > > +++ b/alloc.c
> > > > @@ -29,6 +29,23 @@
> > > > #include "hugetlbfs.h"
> > > > #include "libhugetlbfs_internal.h"
> > > >
> > > > +/* Allocate base pages if huge page allocation fails */
> > > > +static void *fallback_base_pages(size_t len, ghp_t flags)
> > > > +{
> > > > + void *buf;
> > > > + DEBUG("Insufficient free huge pages, falling back to base
> > > > pages\n");
> > > > + buf = mmap(NULL, len,
> > > > + PROT_READ|PROT_WRITE,
> > > > + MAP_PRIVATE|MAP_ANONYMOUS,
> > > > + 0, 0);
> > > > + if (buf == MAP_FAILED) {
> > > > + WARNING("Base page fallback failed: %s\n",
> > > > strerror(errno));
> > > > + buf = NULL;
> > > > + }
> > > > +
> > > > + return buf;
> > > > +}
> > > > +
> > > > /**
> > > > * get_huge_pages - Allocate an amount of memory backed by huge pages
> > > > * len: Size of the region to allocate
> > > > @@ -58,7 +75,13 @@ void *get_huge_pages(size_t len, ghp_t flags)
> > > > MAP_PRIVATE, heap_fd, len);
> > > > if (buf == MAP_FAILED) {
> > > > close(heap_fd);
> > > > - WARNING("New heap segment map failed: %s\n",
> > > > strerror(errno));
> > > > +
> > > > + /* Try falling back to base pages if allowed */
> > > > + if (flags & GHP_FALLBACK)
> > > > + return fallback_base_pages(len, flags);
> > > > +
> > > > + WARNING("New heap segment map (flags: 0x%lX) failed:
> > > > %s\n",
> > > > + flags, strerror(errno));
> > > > return NULL;
> > > > }
> > > >
> > > > diff --git a/hugetlbfs.h b/hugetlbfs.h
> > > > index 3945836..30d1fa1 100644
> > > > --- a/hugetlbfs.h
> > > > +++ b/hugetlbfs.h
> > > > @@ -40,6 +40,7 @@ long dump_proc_pid_maps(void);
> > > > * by the current implementation of the library
> > > > */
> > > > typedef unsigned long ghp_t;
> > > > +#define GHP_FALLBACK (0x01UL)/* Use base pages if necessary
> > > > */
> > > > #define GHP_DEFAULT(0)
> > > >
> > > > /* Direct alloc functions */
> > > > diff --git a/tests/get_huge_pages.c b/tests/get_huge_pages.c
> > > > index b05e849..cc19a73 100644
> > > > --- a/tests/
[Libhugetlbfs-devel] [PATCH 1/2] build: Cleanup tools build V2
To properly support building and installing tools (hugectl, hugeedit, hugecfg, etc), some changes to the initial method of building hugectl are required. This patch creates wildcard rules that can be used for building either 32 or 64 bit versions of all tools. Additionally, a native wordsize is selected for each architecture so that only one binary (of the desired default word size) is built and installed. Changes since V1: - Allow the compiler to choose whether to build 32 or 64 bit tools Signed-off-by: Adam Litke <[EMAIL PROTECTED]> --- Makefile | 16 +--- 1 files changed, 9 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index d69a365..8030d30 100644 --- a/Makefile +++ b/Makefile @@ -2,9 +2,9 @@ PREFIX = /usr/local EXEDIR = /bin LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o -OBJS = hugectl.o INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a -INSTALL_OBJ = hugectl +BIN_OBJ_DIR=obj +INSTALL_BIN = hugectl INSTALL_HEADERS = hugetlbfs.h LDSCRIPT_TYPES = B BDT LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64 @@ -139,7 +139,7 @@ tests: tests/all tests/%: $(MAKE) -C tests $* -tools: $(INSTALL_OBJ) +tools: $(foreach file,$(INSTALL_BIN),$(BIN_OBJ_DIR)/$(file)) check: all cd tests; ./run_tests.sh @@ -230,15 +230,15 @@ $(OBJS): hugectl.c @$(VECHO) CPP $@ $(CC) $(CFLAGS) -o $@ -c $< -$(INSTALL_OBJ):$(OBJS) +$(INSTALL_BIN:%=$(BIN_OBJ_DIR)/%): $(BIN_OBJ_DIR)/%: %.c @$(VECHO) CC $@ - $(CC) $(CFLAGS) -o $@ $< + mkdir -p $(BIN_OBJ_DIR) + $(CC) $(CPPFLAGS) $(CFLAGS) -o $@ $^ clean: @$(VECHO) CLEAN rm -f *~ *.o *.so *.a *.d *.i core a.out $(VERSION) rm -rf obj* - rm -f $(OBJ) $(INSTALL_OBJ) rm -f ldscripts/*~ rm -f libhugetlbfs-sock $(MAKE) -C tests clean @@ -275,7 +275,9 @@ install: libs tools $(OBJDIRS:%=%/install) $(INSTALL_OBJSCRIPT:%=objscript.%) $(INSTALL) -m 644 $(INSTALL_LDSCRIPTS:%=ldscripts/%) $(DESTDIR)$(LDSCRIPTDIR) $(INSTALL) -d $(DESTDIR)$(BINDIR) $(INSTALL) -d $(DESTDIR)$(EXEDIR) - $(INSTALL) -m 755 $(INSTALL_OBJ) $(DESTDIR)$(EXEDIR) + @$(VECHO) INSTALLBIN $(DESTDIR)$(EXEDIR) + for x in $(INSTALL_BIN); do \ + $(INSTALL) -m 755 $(BIN_OBJ_DIR)/$$x $(DESTDIR)$(EXEDIR); done for x in $(INSTALL_OBJSCRIPT); do \ $(INSTALL) -m 755 objscript.$$x $(DESTDIR)$(BINDIR)/$$x; done cd $(DESTDIR)$(BINDIR) && ln -sf ld.hugetlbfs ld - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH] [RFC] Fallback to smaller pages in get_huge_pages() when GHP_FALLBACK is specified V2
get_huge_pages() is an API for the allocation of hugepage-backed regions of
memory. In the event it returns NULL, the application is expected to recover,
possibly by calling mmap() or malloc() to use base pages. This is boiler-plate
code that is an unnecessary burden on the application developer. This patch
adds GHP_FALLBACK to indicate get_huge_pages() should use smaller pages if
hugepages of the default size are not available.
Changelog since V1
o Do not specify HUGETLB_VERBOSE when running the test
o Account for rsvd pages when running the test
o Change definition of GHP_FALLBACK to mean "use smaller pages"
Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
---
alloc.c| 25 -
hugetlbfs.h|3 +++
tests/get_huge_pages.c | 37 +
3 files changed, 64 insertions(+), 1 deletion(-)
diff --git a/alloc.c b/alloc.c
index 65bd063..2a95fb2 100644
--- a/alloc.c
+++ b/alloc.c
@@ -29,6 +29,23 @@
#include "hugetlbfs.h"
#include "libhugetlbfs_internal.h"
+/* Allocate base pages if huge page allocation fails */
+static void *fallback_base_pages(size_t len, ghp_t flags)
+{
+ void *buf;
+ DEBUG("Falling back to base pages\n");
+ buf = mmap(NULL, len,
+ PROT_READ|PROT_WRITE,
+ MAP_PRIVATE|MAP_ANONYMOUS,
+ 0, 0);
+ if (buf == MAP_FAILED) {
+ WARNING("Base page fallback failed: %s\n", strerror(errno));
+ buf = NULL;
+ }
+
+ return buf;
+}
+
/**
* get_huge_pages - Allocate an amount of memory backed by huge pages
* len: Size of the region to allocate
@@ -58,7 +75,13 @@ void *get_huge_pages(size_t len, ghp_t flags)
MAP_PRIVATE, heap_fd, len);
if (buf == MAP_FAILED) {
close(heap_fd);
- WARNING("New heap segment map failed: %s\n", strerror(errno));
+
+ /* Try falling back to base pages if allowed */
+ if (flags & GHP_FALLBACK)
+ return fallback_base_pages(len, flags);
+
+ WARNING("New heap segment map (flags: 0x%lX) failed: %s\n",
+ flags, strerror(errno));
return NULL;
}
diff --git a/hugetlbfs.h b/hugetlbfs.h
index 3945836..91d021f 100644
--- a/hugetlbfs.h
+++ b/hugetlbfs.h
@@ -38,8 +38,11 @@ long dump_proc_pid_maps(void);
*
* GHP_DEFAULT - Use a combination of flags deemed to be a sensible default
* by the current implementation of the library
+ * GHP_FALLBACK - Use the default hugepage size if possible but fallback to
+ * smaller pages if necessary
*/
typedef unsigned long ghp_t;
+#define GHP_FALLBACK (0x01UL)
#define GHP_DEFAULT(0)
/* Direct alloc functions */
diff --git a/tests/get_huge_pages.c b/tests/get_huge_pages.c
index b05e849..12ff387 100644
--- a/tests/get_huge_pages.c
+++ b/tests/get_huge_pages.c
@@ -45,12 +45,49 @@ void test_get_huge_pages(int num_hugepages)
FAIL("hugepage was not correctly freed");
}
+void test_GHP_FALLBACK(void)
+{
+ int err;
+ long hpage_size = check_hugepagesize();
+ long rsvd_hugepages = read_meminfo("HugePages_Rsvd:");
+ long num_hugepages = read_meminfo("HugePages_Total:") - rsvd_hugepages;
+
+ /* We should be able to allocate the whole pool */
+ void *p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
+ if (p == NULL)
+ FAIL("test_GHP_FALLBACK(GHP_DEFAULT) failed for %ld hugepages",
+ num_hugepages);
+ memset(p, 1, hpage_size);
+ err = test_addr_huge(p + (num_hugepages - 1) * hpage_size);
+ if (err != 1)
+ FAIL("Returned page is not hugepage");
+ free_huge_pages(p);
+
+ /* We should fail allocating too much */
+ num_hugepages++;
+ p = get_huge_pages(num_hugepages * hpage_size, GHP_DEFAULT);
+ if (p != NULL)
+ FAIL("test_GHP_FALLBACK() for %ld expected fail, got success",
num_hugepages);
+
+ /* GHP_FALLBACK should succeed by allocating base pages */
+ p = get_huge_pages(num_hugepages * hpage_size, GHP_FALLBACK);
+ if (p == NULL)
+ FAIL("test_GHP_FALLBACK(GHP_FALLBACK) failed for %ld hugepages",
+ num_hugepages);
+ memset(p, 1, hpage_size);
+ err = test_addr_huge(p + (num_hugepages - 1) * hpage_size);
+ if (err == 1)
+ FAIL("Returned page is not a base page");
+ free_huge_pages(p);
+}
+
int main(int argc, char *argv[])
{
test_init(argc, argv);
check_free_huge_pages(4);
test_get_huge_pages(1);
test_get_huge_pages(4);
+ test_GHP_FALLBACK();
PASS();
}
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applicatio
Re: [Libhugetlbfs-devel] [PATCH 2/3] hugectl: convert exit status to POSIX standard codes
On Thu, 24 Jul 2008, Andy Whitcroft wrote: > Move to using the POSIX standard codes EXIT_FAILURE/EXIT_SUCCESS instead > of explicit numbers. > > Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]> Acked-by: Eric B Munson <[EMAIL PROTECTED]> signature.asc Description: Digital signature - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 1/2] build: Cleanup tools build V2
On Thu, Jul 24, 2008 at 03:45:01PM +, Adam Litke wrote: > To properly support building and installing tools (hugectl, hugeedit, hugecfg, > etc), some changes to the initial method of building hugectl are required. > This patch creates wildcard rules that can be used for building either 32 or > 64 > bit versions of all tools. Additionally, a native wordsize is selected for > each architecture so that only one binary (of the desired default word size) > is > built and installed. > > Changes since V1: > - Allow the compiler to choose whether to build 32 or 64 bit tools > > Signed-off-by: Adam Litke <[EMAIL PROTECTED]> > --- > > Makefile | 16 +--- > 1 files changed, 9 insertions(+), 7 deletions(-) > > > diff --git a/Makefile b/Makefile > index d69a365..8030d30 100644 > --- a/Makefile > +++ b/Makefile > @@ -2,9 +2,9 @@ PREFIX = /usr/local > EXEDIR = /bin > > LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o > -OBJS = hugectl.o > INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a > -INSTALL_OBJ = hugectl > +BIN_OBJ_DIR=obj > +INSTALL_BIN = hugectl > INSTALL_HEADERS = hugetlbfs.h > LDSCRIPT_TYPES = B BDT > LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64 > @@ -139,7 +139,7 @@ tests:tests/all > tests/%: > $(MAKE) -C tests $* > > -tools: $(INSTALL_OBJ) > +tools: $(foreach file,$(INSTALL_BIN),$(BIN_OBJ_DIR)/$(file)) > > check: all > cd tests; ./run_tests.sh > @@ -230,15 +230,15 @@ $(OBJS):hugectl.c > @$(VECHO) CPP $@ > $(CC) $(CFLAGS) -o $@ -c $< > > -$(INSTALL_OBJ): $(OBJS) > +$(INSTALL_BIN:%=$(BIN_OBJ_DIR)/%): $(BIN_OBJ_DIR)/%: %.c > @$(VECHO) CC $@ > - $(CC) $(CFLAGS) -o $@ $< > + mkdir -p $(BIN_OBJ_DIR) > + $(CC) $(CPPFLAGS) $(CFLAGS) -o $@ $^ > > clean: > @$(VECHO) CLEAN > rm -f *~ *.o *.so *.a *.d *.i core a.out $(VERSION) > rm -rf obj* > - rm -f $(OBJ) $(INSTALL_OBJ) > rm -f ldscripts/*~ > rm -f libhugetlbfs-sock > $(MAKE) -C tests clean > @@ -275,7 +275,9 @@ install: libs tools $(OBJDIRS:%=%/install) > $(INSTALL_OBJSCRIPT:%=objscript.%) > $(INSTALL) -m 644 $(INSTALL_LDSCRIPTS:%=ldscripts/%) > $(DESTDIR)$(LDSCRIPTDIR) > $(INSTALL) -d $(DESTDIR)$(BINDIR) > $(INSTALL) -d $(DESTDIR)$(EXEDIR) > - $(INSTALL) -m 755 $(INSTALL_OBJ) $(DESTDIR)$(EXEDIR) > + @$(VECHO) INSTALLBIN $(DESTDIR)$(EXEDIR) > + for x in $(INSTALL_BIN); do \ > + $(INSTALL) -m 755 $(BIN_OBJ_DIR)/$$x $(DESTDIR)$(EXEDIR); done > for x in $(INSTALL_OBJSCRIPT); do \ > $(INSTALL) -m 755 objscript.$$x $(DESTDIR)$(BINDIR)/$$x; done > cd $(DESTDIR)$(BINDIR) && ln -sf ld.hugetlbfs ld That is exactly what I had in mind. Dunno if I have an Ack, but LGTM. Acked-by: Andy Whitcroft <[EMAIL PROTECTED]> Tested-by: Andy Whitcroft <[EMAIL PROTECTED]> -apw - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 1/2] build: Cleanup tools build
On Thu, 2008-07-24 at 10:24 +0100, Andy Whitcroft wrote: > I am a little confused by why we consider bit size at all for the 'host' > applications. We seem to build one or other only for host applications > even where the host supports both sizes. But the makefile has to specify > which they are. Could we not just have 'objnative' (perhaps just 'obj' in > line with obj32/64) for native host applications, and just always compile > them without any sort of -mXX, with $(CC). Clearly the applications are > compilable as either size as we do that for different architectures? Yeah. I really like this idea. I'll update the patch and do it this way. -- Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH] Allow shmget() to be overridden to add the SHM_HUGETLB flag V3
On Tue, 2008-07-22 at 18:42 +0100, Mel Gorman wrote:
> +int shmget(key_t key, size_t size, int shmflg)
> +{
> + static int (*real_shmget)(key_t key, size_t size, int shmflg) = NULL;
> + void *handle;
> + char *error;
> + int retval;
> + char *hugetlbshm_env;
> + size_t aligned_size = size;
> + int hugetlbshm_enabled = 0;
> +
> + DEBUG("hugetlb_shmem: using overridden shmget() call\n");
> +
> + /* Get a handle to the "real" shmget system call */
> + if (!real_shmget) {
> + handle = dlopen(LIBC, RTLD_LAZY);
> + if (!handle) {
> + ERROR("%s", dlerror());
> + return -1;
> + }
> + real_shmget = dlsym(handle, "shmget");
> + if ((error = dlerror()) != NULL) {
> + ERROR("%s", error);
> + return -1;
> + }
> + }
commit 7eab70f316f531454234bf7b54cba0950a73675e
Author: Adam Litke <[EMAIL PROTECTED]>
Date: Thu Jul 24 16:31:20 2008 +
It is actually a bit easier to use dlsym() to override shmget than what you
are
currently doing. Rather than using dlopen to bring in a specific library,
we
can use the RTLD_NEXT psuedo-handle when calling dlsym.
From the dlsym man page:
There are two special pseudo-handles, RTLD_DEFAULT and RTLD_NEXT.
The
former will find the first occurrence of the desired symbol using
the
default library search order. The latter will find the next
occurrence
of a function in the search order after the current library.
This
allows one to provide a wrapper around a function in another
shared
library.
Doing this will eliminate the need to hardcode the libc library into the
build
process.
Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
diff --git a/Makefile b/Makefile
index 0f3b436..97c2c25 100644
--- a/Makefile
+++ b/Makefile
@@ -13,12 +13,11 @@ VERSION=version.h
SOURCE = $(shell find . -maxdepth 1 ! -name version.h -a -name '*.[h]')
SOURCE += *.c *.lds Makefile
NODEPTARGETS=
-LIBC = $(shell ldd /bin/ls | grep libc | awk '{print $$3}')
INSTALL = install
LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds -ldl
-CFLAGS ?= -O2 -g -DLIBC="\"$(LIBC)\""
+CFLAGS ?= -O2 -g
CFLAGS += -Wall -fPIC
CPPFLAGS += -D__LIBHUGETLBFS__
diff --git a/shm.c b/shm.c
index 282acaf..48be250 100755
--- a/shm.c
+++ b/shm.c
@@ -16,6 +16,7 @@
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*/
+#define _GNU_SOURCE
#include
#include
#include
@@ -42,12 +43,7 @@ int shmget(key_t key, size_t size, int shmflg)
/* Get a handle to the "real" shmget system call */
if (!real_shmget) {
- handle = dlopen(LIBC, RTLD_LAZY);
- if (!handle) {
- ERROR("%s", dlerror());
- return -1;
- }
- real_shmget = dlsym(handle, "shmget");
+ real_shmget = dlsym(RTLD_NEXT, "shmget");
if ((error = dlerror()) != NULL) {
ERROR("%s", error);
return -1;
> diff --git a/tests/shmoverride_unlinked.c b/tests/shmoverride_unlinked.c
For the life of me, I can't get this test to pass on 64bit ppc. I get
the following error and I can't really tell what it means.
LD_PRELOAD=libhugetlbfs.so shmoverride_unlinked (64): FAIL
override-requested: Badness expected 1, actual 0
> new file mode 100644
> index 000..583becf
> --- /dev/null
> +++ b/tests/shmoverride_unlinked.c
--
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 2/2] tools: hugeedit: Modify binaries to set default remapping behavior V4
On Thu, Jul 24, 2008 at 03:45:08PM +, Adam Litke wrote:
> The recently added relinking method makes it possible to choose from multiple
> ways to remap segments at run-time via the HUGTLB_ELFMAP environment variable.
> If nothing is specified in that variable, then no remapping will occur.
> Sometimes it is desirable to set a default remapping mode so that
> HUGETLB_ELFMAP does not need to be set. This semantic applied to the original
> remapping algorithm.
>
> This patch adds a utility that can modify the PF_LINUX_HUGETLB segment flags
> in
> the ELF haeder of a binary to control which segments will be remapped by
> default. The tool will also display this information.
>
> Changes since V3 (Thanks Andy)
> - Use REPORT infrastructure for error messages
> - --disable cannot be specified with --text and/or --data
> - Usage message macros
> - Check return values of munmap and close to avoid silent data loss
> - Whitespace cleanups
>
> Changes since V2 (Thanks again Mel)
> - Support for remapping more of the ELF header if one page is not enough
> - is_text() checks that p_filesz == p_memsz in addition to permissions
> - Rename scan_phdrs* to update_phdrs*
> - Print segment start and end addresses
> - Whitespace
>
> Changes since V1 (Thanks Mel for the review):
> - New --text, --data, and --disable options
> - Binary default modes are always displayed
> - Binary is opened for writing only when updates have been requested
>
> Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
> ---
>
> Makefile|2
> hugeedit.c | 239
> +++
> libhugetlbfs_internal.h |2
> 3 files changed, 242 insertions(+), 1 deletions(-)
> create mode 100644 hugeedit.c
>
>
> diff --git a/Makefile b/Makefile
> index 8030d30..f1f83fa 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -4,7 +4,7 @@ EXEDIR = /bin
> LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o
> INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
> BIN_OBJ_DIR=obj
> -INSTALL_BIN = hugectl
> +INSTALL_BIN = hugectl hugeedit
> INSTALL_HEADERS = hugetlbfs.h
> LDSCRIPT_TYPES = B BDT
> LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64
> diff --git a/hugeedit.c b/hugeedit.c
> new file mode 100644
> index 000..3bfcfe3
> --- /dev/null
> +++ b/hugeedit.c
> @@ -0,0 +1,239 @@
> +/*
> + * libhugetlbfs - Easy use of Linux hugepages
> + * Copyright (C) 2008 Adam Litke, IBM Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public License
> + * as published by the Free Software Foundation; either version 2.1 of
> + * the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#define _GNU_SOURCE
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +
> +/*
> + * Eventually we plan to use the libhugetlbfs reporting facility,
> + * but until that is possible, redefine a simpler version here.
> + */
> +#define REPORT(level, prefix, format, ...) \
> + do { \
> + fprintf(stderr, "hugeedit: " prefix ": " format, \
> + ##__VA_ARGS__); \
> + } while (0)
> +
> +#include "libhugetlbfs_internal.h"
> +
> +/*
> + * All MAP_* options are tagged with MAP_BASE to differentiate them as
> options
> + * in the options parser. This must be removed before they are compared.
> + */
> +#define MAP_BASE 0x1000
> +#define MAP_DISABLE 0x0001
> +#define MAP_TEXT 0x0002
> +#define MAP_DATA 0x0004
> +
> +#define PF_LINUX_HUGETLB 0x10
> +extern int optind;
> +extern char *optarg;
> +
> +#define OPTION(opts, text) fprintf(stderr, " %-25s %s\n", opts, text)
> +#define CONT(text) fprintf(stderr, " %-25s %s\n", "", text)
> +
> +void print_usage()
> +{
> + printf("hugeedit [options] target\n");
> + printf("options:\n");
> + OPTION("--text", "Remap program text into huge pages by default");
> + OPTION("--data", "Remap program data into huge pages by default");
> + OPTION("--disable", "Remap no segments into huge pages by default");
> + OPTION("--help", "Print this usage information");
Mention the short form of this one, this is how I did it in hugectl:
OPTION("--help, -h", "Print this usage information");
> +}
> +
> +int check_elf_wordsize(void *ehdr)
> +{
> + char *e_ident = (char *) ehdr;
> +
Re: [Libhugetlbfs-devel] [PATCH] Allow shmget() to be overridden to add the SHM_HUGETLB flag V3
On (24/07/08 11:35), Adam Litke didst pronounce:
> On Tue, 2008-07-22 at 18:42 +0100, Mel Gorman wrote:
> > +int shmget(key_t key, size_t size, int shmflg)
> > +{
> > + static int (*real_shmget)(key_t key, size_t size, int shmflg) = NULL;
> > + void *handle;
> > + char *error;
> > + int retval;
> > + char *hugetlbshm_env;
> > + size_t aligned_size = size;
> > + int hugetlbshm_enabled = 0;
> > +
> > + DEBUG("hugetlb_shmem: using overridden shmget() call\n");
> > +
> > + /* Get a handle to the "real" shmget system call */
> > + if (!real_shmget) {
> > + handle = dlopen(LIBC, RTLD_LAZY);
> > + if (!handle) {
> > + ERROR("%s", dlerror());
> > + return -1;
> > + }
> > + real_shmget = dlsym(handle, "shmget");
> > + if ((error = dlerror()) != NULL) {
> > + ERROR("%s", error);
> > + return -1;
> > + }
> > + }
>
> commit 7eab70f316f531454234bf7b54cba0950a73675e
> Author: Adam Litke <[EMAIL PROTECTED]>
> Date: Thu Jul 24 16:31:20 2008 +
>
> It is actually a bit easier to use dlsym() to override shmget than what
> you are
> currently doing. Rather than using dlopen to bring in a specific
> library, we
> can use the RTLD_NEXT psuedo-handle when calling dlsym.
>
> From the dlsym man page:
>
>There are two special pseudo-handles, RTLD_DEFAULT and RTLD_NEXT.
> The
>former will find the first occurrence of the desired symbol using
> the
>default library search order. The latter will find the next
> occurrence
>of a function in the search order after the current library.
> This
>allows one to provide a wrapper around a function in another
> shared
>library.
>
> Doing this will eliminate the need to hardcode the libc library into the
> build
> process.
>
> Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
>
> diff --git a/Makefile b/Makefile
> index 0f3b436..97c2c25 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -13,12 +13,11 @@ VERSION=version.h
> SOURCE = $(shell find . -maxdepth 1 ! -name version.h -a -name '*.[h]')
> SOURCE += *.c *.lds Makefile
> NODEPTARGETS=
> -LIBC = $(shell ldd /bin/ls | grep libc | awk '{print $$3}')
>
> INSTALL = install
>
> LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds -ldl
> -CFLAGS ?= -O2 -g -DLIBC="\"$(LIBC)\""
> +CFLAGS ?= -O2 -g
> CFLAGS += -Wall -fPIC
> CPPFLAGS += -D__LIBHUGETLBFS__
>
> diff --git a/shm.c b/shm.c
> index 282acaf..48be250 100755
> --- a/shm.c
> +++ b/shm.c
> @@ -16,6 +16,7 @@
> * License along with this library; if not, write to the Free Software
> * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> */
> +#define _GNU_SOURCE
> #include
> #include
> #include
> @@ -42,12 +43,7 @@ int shmget(key_t key, size_t size, int shmflg)
>
> /* Get a handle to the "real" shmget system call */
> if (!real_shmget) {
> - handle = dlopen(LIBC, RTLD_LAZY);
> - if (!handle) {
> - ERROR("%s", dlerror());
> - return -1;
> - }
> - real_shmget = dlsym(handle, "shmget");
> + real_shmget = dlsym(RTLD_NEXT, "shmget");
> if ((error = dlerror()) != NULL) {
> ERROR("%s", error);
> return -1;
>
Ah, very cool. That does simplify things a bit and makes it a lot less
sensitive to upgrades to glibc. Thanks.
>
>
> > diff --git a/tests/shmoverride_unlinked.c b/tests/shmoverride_unlinked.c
>
> For the life of me, I can't get this test to pass on 64bit ppc. I get
> the following error and I can't really tell what it means.
>
> LD_PRELOAD=libhugetlbfs.so shmoverride_unlinked (64): FAIL
> override-requested: Badness expected 1, actual 0
>
It implies that the preload is not triggering properly because 1
hugepage should have been used up and it wasn't. I'll check it out
closer.
> > new file mode 100644
> > index 000..583becf
> > --- /dev/null
> > +++ b/tests/shmoverride_unlinked.c
>
> --
> Adam Litke - (agl at us.ibm.com)
> IBM Linux Technology Center
>
>
> -
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> ___
> Libhugetlbfs-devel mailing list
> Libhugetlbfs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
>
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Soft
[Libhugetlbfs-devel] [PATCH 2/6] hugectl: add remap support via --text, --data, --bss, and --disable
Add control over remap via the --text, --data, --bss, and --disable
options. The first three request mapping of those segments, the last
disables all remap. Where the combinations requested cannot be exactly
handled then the request is "widened" to get that segment remapped,
for example if you request --data or --bss in isolation then both are
remapped and a warning emitted.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
---
hugectl.c | 71 +
1 files changed, 71 insertions(+), 0 deletions(-)
diff --git a/hugectl.c b/hugectl.c
index ffbd260..0387c5b 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -58,6 +58,10 @@ void print_usage()
fprintf(stderr, "options:\n");
OPTION("--help, -h", "Prints this message");
+
+ OPTION("--text", "Requests remapping of the program text");
+ OPTION("--data", "Requests remapping of the program data");
+ OPTION("--bss", "Requests remapping of the program bss");
}
int verbose_level = VERBOSITY_DEFAULT;
@@ -74,12 +78,71 @@ void verbose_init(void)
verbose_level = VERBOSITY_MAX;
}
+void setup_environment(char *var, char *val)
+{
+ setenv(var, val, 1);
+ DEBUG("%s='%s'\n", var, val);
+}
+
+
+/*
+ * getopts return values for options which are long only.
+ */
+#define MAP_BASE 0x1000
+
+/*
+ * Mapping selectors, one bit per remappable/backable area as requested
+ * by the user. These are also used as returns from getopts where they
+ * are offset from MAP_BASE, which must be removed before they are compared.
+ */
+#define MAP_DISABLE0x0001
+#define MAP_TEXT 0x0002
+#define MAP_DATA 0x0004
+#define MAP_BSS0x0008
+
+void setup_mappings(int which)
+{
+ char remap[3] = { 0, 0, 0 };
+ int n = 0;
+
+ /*
+* HUGETLB_ELFMAP should be set to either a combination of 'R' and 'W'
+* which indicate which segments should be remapped. It may also be
+* set to 'no' to prevent remapping.
+*/
+ if (which & MAP_TEXT)
+ remap[n++] = 'R';
+ if (which & (MAP_DATA|MAP_BSS)) {
+ if ((which & (MAP_DATA|MAP_BSS)) != (MAP_DATA|MAP_BSS))
+ WARNING("data and bss remapped together\n");
+ remap[n++] = 'W';
+ }
+ if (which & MAP_DISABLE) {
+ if (which != MAP_DISABLE)
+ WARNING("--disable masks requested remap\n");
+ n = 0;
+ remap[n++] = 'n';
+ remap[n++] = 'o';
+ }
+
+ if (n)
+ setup_environment("HUGETLB_ELFMAP", remap);
+}
+
int main(int argc, char** argv)
{
+ int opt_mappings = 0;
+
char opts[] = "+h";
int ret = 0, index = 0;
struct option long_opts[] = {
{"help", no_argument, NULL, 'h'},
+
+ {"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
+ {"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
+ {"data", no_argument, NULL, MAP_BASE|MAP_DATA},
+ {"bss",no_argument, NULL, MAP_BASE|MAP_BSS},
+
{0},
};
@@ -87,6 +150,10 @@ int main(int argc, char** argv)
while (ret != -1) {
ret = getopt_long(argc, argv, opts, long_opts, &index);
+ if (ret > 0 && (ret & MAP_BASE)) {
+ opt_mappings |= ret;
+ continue;
+ }
switch (ret) {
case '?':
print_usage();
@@ -103,12 +170,16 @@ int main(int argc, char** argv)
}
}
index = optind;
+ opt_mappings &= ~MAP_BASE;
if ((argc - index) < 1) {
print_usage();
exit(EXIT_FAILURE);
}
+ if (opt_mappings)
+ setup_mappings(opt_mappings);
+
execvp(argv[index], &argv[index]);
ERROR("exec failed: %s\n", strerror(errno));
exit(EXIT_FAILURE);
--
1.5.6.GIT
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 0/6] [V4] hugectl: remap control
This is a further revision of the basic hugectl support incorporating further feedback, and a number of other cleanups which came out of the work to address them. This is based on my cleanup stack 'hugectl cleanups II (V1)'. Things seem to be stabalising here. Needing resolution: 1) are we happy with the command line form, as its essentially an ABI once released, and 2) is PRELOAD actually always safe if so we can just enable it. There was confusion about whether the linking order would be broken by it. Changelog: V4: - separate out cleanups into their own series - all new option help specification - option --explain becomes --dry-run - option --no-library becomes --library-use-path V3: - abstract out the verbosity levels - cleanup and abstract long option values - always indicate environmental variables as they are set - library path defaults to set, and may be suppressed - adds --explain to expose configuration V2: - incorporate community feedback I have been looking at adding basic functionality to hugectl. Following on from my cleanup series this patch set adds basic remap functionality to hugectl. The short version of the story is that it adds options to request backing of the text, data, bss, and heap as first class options and simplifies preload handling, library path specification, and exposes configuration on request. The main driver for hugectl is to simplify the use of hugepages as exported through the libhugetlbfs library. Specifically we should be trying to do what the user means where possible. This patch set brings basic support for requesting remap and/or backing of the various elements program elements supported by libhugetlbfs. This includes the request of remap of the various program segments, as well as requesting the heap (malloc space) be backed. Where appropriate it tries to configure preload of the library. When considering the current configuration the clearest anomoly seems to be the difference in handling of normal program segments and malloc. From a user point of view hugepages are something which I use for something, to back something. Backing my text with hugepages or my malloc space with hugepages are the same thing. That is I have a bunch of things, text, data, bss, malloc all of which I may wish or not wish to back with hugepages. The underlying implementation differences are hidden by exposing each as its own option, all at the same level. hugectl --text hugectl --data --bss hugectl --heap hugectl --stack (for illustrative purposes only) hugectl --disable Note that while each thing is separately specified, in reality only certain combinations are valid. Where the user specifies a bad combination it is "widened" to a valid combination which includes their request and a warning is emitted. This helps to get the user going and is part of the DWIM approach. Preloading is a little tricky as it is only useful where the binary is not-prelinked and only there is we are only asking for the heap to be backed. But in that combination we can always preload safely which should remove the need to every conciously request it. In case it is a problem we allow it to be suppressed via --no-preload. Finally it is helpful to be able to see the options in use for any particular combination of arguments, expose this via --explain. This stack consists of 6 patches: hugectl: honour the hugetlb verbosity/debug flags -- reads the standard libhugetlbfs verbosity and debug control flags and supplies compatible error and warning routines, hugectl: add remap support via --text, --data, --bss, and --disable -- which adds basic remap support via separated options, hugectl: add backing of the heap via --heap -- which adds heap remapping via --heap, hugectl: add support for preload -- where preload is appropriate request preloading of the library, hugectl: add support for requesting which library to use -- providing an override for the normal library path allowing clean execution in non-standard installs, and hugectl: explain the hugetlb configuration selected via --explain -- which allow exposure of the variable settings for a specific command line. Note that the first of these patches does duplicate some functionality from the main library and that will be sorted out as a separate series. Thanks for Nish, Adam, Mel, David and Andrew for feedback during this process. Comments? -apw Andy Whitcroft (6): hugectl: honour the hugetlb verbosity/debug flags hugectl: add remap support via --text, --data, --bss, and --disable hugectl: add backing of the heap via --heap hugectl: add support for preload hugectl: add support for requesting which library to use hugectl: add a --dry-run option to dump what would be done Makefile |9 ++- hugectl.c | 226 - 2 files changed, 231 insertions(+), 4 deletions(-)
[Libhugetlbfs-devel] [PATCH 6/6] hugectl: add a --dry-run option to dump what would be done
Allow the user to request the specific hugetlb configuration which will
be used for the selected options, exposed via the --dry-run option.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
---
hugectl.c | 18 +-
1 files changed, 17 insertions(+), 1 deletions(-)
diff --git a/hugectl.c b/hugectl.c
index 81e76b2..f369908 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -78,8 +78,11 @@ void print_usage()
LIBDIR32 ":"
#endif
")");
+
+ OPTION("--dry-run", "describe what would be done without doing it");
}
+int opt_dry_run = 0;
int verbose_level = VERBOSITY_DEFAULT;
void verbose_init(void)
@@ -98,6 +101,9 @@ void setup_environment(char *var, char *val)
{
setenv(var, val, 1);
DEBUG("%s='%s'\n", var, val);
+
+ if (opt_dry_run)
+ printf("%s='%s'\n", var, val);
}
@@ -112,6 +118,8 @@ void setup_environment(char *var, char *val)
#define LONG_NO_LIBRARY(LONG_BASE | 'L')
#define LONG_LIBRARY (LONG_BASE | 'l')
+#define LONG_DRY_RUN (LONG_BASE | 'd')
+
/*
* Mapping selectors, one bit per remappable/backable area as requested
* by the user. These are also used as returns from getopts where they
@@ -224,6 +232,7 @@ int main(int argc, char** argv)
{"library-use-path",
no_argument, NULL, LONG_NO_LIBRARY},
{"library",required_argument, NULL, LONG_LIBRARY},
+ {"dry-run",no_argument, NULL, LONG_DRY_RUN},
{"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
{"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
@@ -265,6 +274,10 @@ int main(int argc, char** argv)
opt_library = optarg;
break;
+ case LONG_DRY_RUN:
+ opt_dry_run = 1;
+ break;
+
default:
WARNING("unparsed option %08x\n", ret);
ret = -1;
@@ -274,7 +287,7 @@ int main(int argc, char** argv)
index = optind;
opt_mappings &= ~MAP_BASE;
- if ((argc - index) < 1) {
+ if (!opt_dry_run && (argc - index) < 1) {
print_usage();
exit(EXIT_FAILURE);
}
@@ -288,6 +301,9 @@ int main(int argc, char** argv)
if (opt_preload)
ldpreload(opt_mappings);
+ if (opt_dry_run)
+ exit(EXIT_SUCCESS);
+
execvp(argv[index], &argv[index]);
ERROR("exec failed: %s\n", strerror(errno));
exit(EXIT_FAILURE);
--
1.5.6.GIT
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 1/6] hugectl: honour the hugetlb verbosity/debug flags
Handle the HUGETLB_VERBOSE and HUGETLB_DEBUG flags such that we will emit
messages at the same levels as defined for the library. Use the common
header to define the levels. Convert all existing errors to use the new
macros.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
Acked-by: Mel Gorman <[EMAIL PROTECTED]>
---
hugectl.c | 29 +++--
1 files changed, 27 insertions(+), 2 deletions(-)
diff --git a/hugectl.c b/hugectl.c
index 8ba794c..ffbd260 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -36,6 +36,15 @@
#include
#include
+#define REPORT(level, prefix, format, ...) \
+ do { \
+ if (verbose_level >= level) \
+ fprintf(stderr, "hugectl: " prefix ": " format, \
+ ##__VA_ARGS__); \
+ } while (0);
+
+#include "libhugetlbfs_debug.h"
+
extern int errno;
extern int optind;
extern char *optarg;
@@ -51,6 +60,20 @@ void print_usage()
OPTION("--help, -h", "Prints this message");
}
+int verbose_level = VERBOSITY_DEFAULT;
+
+void verbose_init(void)
+{
+ char *env;
+
+ env = getenv("HUGETLB_VERBOSE");
+ if (env)
+ verbose_level = atoi(env);
+ env = getenv("HUGETLB_DEBUG");
+ if (env)
+ verbose_level = VERBOSITY_MAX;
+}
+
int main(int argc, char** argv)
{
char opts[] = "+h";
@@ -60,6 +83,8 @@ int main(int argc, char** argv)
{0},
};
+ verbose_init();
+
while (ret != -1) {
ret = getopt_long(argc, argv, opts, long_opts, &index);
switch (ret) {
@@ -72,7 +97,7 @@ int main(int argc, char** argv)
exit(EXIT_SUCCESS);
default:
- fprintf(stderr, "unparsed option %08x\n", ret);
+ WARNING("unparsed option %08x\n", ret);
ret = -1;
break;
}
@@ -85,6 +110,6 @@ int main(int argc, char** argv)
}
execvp(argv[index], &argv[index]);
- fprintf(stderr, "Error calling execvp: '%s'\n", strerror(errno));
+ ERROR("exec failed: %s\n", strerror(errno));
exit(EXIT_FAILURE);
}
--
1.5.6.GIT
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 3/6] hugectl: add backing of the heap via --heap
Add support for mapping the heap via morecore through the --heap option.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
Acked-by: Mel Gorman <[EMAIL PROTECTED]>
---
hugectl.c |7 +++
1 files changed, 7 insertions(+), 0 deletions(-)
diff --git a/hugectl.c b/hugectl.c
index 0387c5b..79e934c 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -62,6 +62,8 @@ void print_usage()
OPTION("--text", "Requests remapping of the program text");
OPTION("--data", "Requests remapping of the program data");
OPTION("--bss", "Requests remapping of the program bss");
+ OPTION("--heap", "Requests remapping of the program heap");
+ CONT("(malloc space)");
}
int verbose_level = VERBOSITY_DEFAULT;
@@ -99,6 +101,7 @@ void setup_environment(char *var, char *val)
#define MAP_TEXT 0x0002
#define MAP_DATA 0x0004
#define MAP_BSS0x0008
+#define MAP_HEAP 0x0010
void setup_mappings(int which)
{
@@ -127,6 +130,9 @@ void setup_mappings(int which)
if (n)
setup_environment("HUGETLB_ELFMAP", remap);
+
+ if (which & MAP_HEAP)
+ setup_environment("HUGETLB_MORECORE", "yes");
}
int main(int argc, char** argv)
@@ -142,6 +148,7 @@ int main(int argc, char** argv)
{"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
{"data", no_argument, NULL, MAP_BASE|MAP_DATA},
{"bss",no_argument, NULL, MAP_BASE|MAP_BSS},
+ {"heap", no_argument, NULL, MAP_BASE|MAP_HEAP},
{0},
};
--
1.5.6.GIT
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 4/6] hugectl: add support for preload
Preload is only useful when linking against an non-relinked binary
and only then when we are only attempting to back the heap. No other
combination can usefully use preload. Using preload with a relinked
binary is safe as long as you are only requesting backing of the heap.
Therefore unconditionally enable preload iff we are requesting only --heap.
As a failsafe allow the user to disable preload.
Note that we make use of the fact that for a standard install the
libraries will appear on the default libarary path, and that the loader
will correctly pick an appropriate library 32/64 bit based on the binary.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
Acked-by: Mel Gorman <[EMAIL PROTECTED]>
---
hugectl.c | 25 +
1 files changed, 25 insertions(+), 0 deletions(-)
diff --git a/hugectl.c b/hugectl.c
index 79e934c..755d3b6 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -64,6 +64,8 @@ void print_usage()
OPTION("--bss", "Requests remapping of the program bss");
OPTION("--heap", "Requests remapping of the program heap");
CONT("(malloc space)");
+
+ OPTION("--no-preload", "Disable preloading the libhugetlbfs library");
}
int verbose_level = VERBOSITY_DEFAULT;
@@ -91,6 +93,9 @@ void setup_environment(char *var, char *val)
* getopts return values for options which are long only.
*/
#define MAP_BASE 0x1000
+#define LONG_BASE 0x2000
+
+#define LONG_NO_PRELOAD(LONG_BASE | 'p')
/*
* Mapping selectors, one bit per remappable/backable area as requested
@@ -135,14 +140,26 @@ void setup_mappings(int which)
setup_environment("HUGETLB_MORECORE", "yes");
}
+void ldpreload(int which)
+{
+ if (which == MAP_HEAP) {
+ setup_environment("LD_PRELOAD", "libhugetlbfs.so");
+ WARNING("LD_PRELOAD in use for lone --heap\n");
+ } else {
+ DEBUG("LD_PRELOAD not appropriate for this map combination\n");
+ }
+}
+
int main(int argc, char** argv)
{
int opt_mappings = 0;
+ int opt_preload = 1;
char opts[] = "+h";
int ret = 0, index = 0;
struct option long_opts[] = {
{"help", no_argument, NULL, 'h'},
+ {"no-preload", no_argument, NULL, LONG_NO_PRELOAD},
{"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
{"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
@@ -170,6 +187,11 @@ int main(int argc, char** argv)
print_usage();
exit(EXIT_SUCCESS);
+ case LONG_NO_PRELOAD:
+ opt_preload = 0;
+ DEBUG("LD_PRELOAD disabled\n");
+ break;
+
default:
WARNING("unparsed option %08x\n", ret);
ret = -1;
@@ -187,6 +209,9 @@ int main(int argc, char** argv)
if (opt_mappings)
setup_mappings(opt_mappings);
+ if (opt_preload)
+ ldpreload(opt_mappings);
+
execvp(argv[index], &argv[index]);
ERROR("exec failed: %s\n", strerror(errno));
exit(EXIT_FAILURE);
--
1.5.6.GIT
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 5/6] hugectl: add support for requesting which library to use
Add support for requesting a specific library set for preload. This adds
the --library option which takes an optional argument. Without an argument
it requests use of the specific libraries installed with the version
of hugectl in use. An argument is treated as a prefix to the library
directories and the approprate 32/64 bit library directories are added.
Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
---
Makefile |9 ++-
hugectl.c | 76 +
2 files changed, 84 insertions(+), 1 deletions(-)
diff --git a/Makefile b/Makefile
index d69a365..6571672 100644
--- a/Makefile
+++ b/Makefile
@@ -101,6 +101,13 @@ BINDIR = $(PREFIX)/share/libhugetlbfs
EXEDIR = $(PREFIX)/bin
DOCDIR = $(PREFIX)/share/doc/libhugetlbfs
+ifdef LIB32
+LIBPATHS += -DLIB32='"$(LIB32)"' -DLIBDIR32='"$(LIBDIR32)"'
+endif
+ifdef LIB64
+LIBPATHS += -DLIB64='"$(LIB64)"' -DLIBDIR64='"$(LIBDIR64)"'
+endif
+
EXTRA_DIST = \
README \
HOWTO \
@@ -228,7 +235,7 @@ obj64/%.s: %.c
$(OBJS): hugectl.c
@$(VECHO) CPP $@
- $(CC) $(CFLAGS) -o $@ -c $<
+ $(CC) $(CFLAGS) $(LIBPATHS) -o $@ -c $<
$(INSTALL_OBJ):$(OBJS)
@$(VECHO) CC $@
diff --git a/hugectl.c b/hugectl.c
index 755d3b6..81e76b2 100644
--- a/hugectl.c
+++ b/hugectl.c
@@ -31,6 +31,7 @@
#include
#include
#include
+#include
#define _GNU_SOURCE /* for getopt_long */
#include
@@ -66,6 +67,17 @@ void print_usage()
CONT("(malloc space)");
OPTION("--no-preload", "Disable preloading the libhugetlbfs library");
+
+ OPTION("--library-use-path", "Use the system library path");
+ OPTION("--library ", "Select a library prefix");
+ CONT("(Default: "
+#ifdef LIBDIR32
+ LIBDIR32 ":"
+#endif
+#ifdef LIBDIR32
+ LIBDIR32 ":"
+#endif
+ ")");
}
int verbose_level = VERBOSITY_DEFAULT;
@@ -97,6 +109,9 @@ void setup_environment(char *var, char *val)
#define LONG_NO_PRELOAD(LONG_BASE | 'p')
+#define LONG_NO_LIBRARY(LONG_BASE | 'L')
+#define LONG_LIBRARY (LONG_BASE | 'l')
+
/*
* Mapping selectors, one bit per remappable/backable area as requested
* by the user. These are also used as returns from getopts where they
@@ -140,6 +155,51 @@ void setup_mappings(int which)
setup_environment("HUGETLB_MORECORE", "yes");
}
+#define LIBRARY_DISABLE ((void *)-1)
+
+void library_path(char *path)
+{
+ char val[NAME_MAX] = "";
+ char *env;
+
+ /*
+* Select which libraries we wish to use. No argument implies
+* use the libraries included with this version of hugectl.
+* Else it is assumed to be a prefix to the 32/64 bit directories
+* both of which are added, where available.
+*/
+ env = getenv("LD_LIBRARY_PATH");
+ if (!path) {
+ /* [LIBDIR32:][LIBDIR64:]$LD_LIBRARY_PATH */
+ snprintf(val, sizeof(val), ""
+#ifdef LIBDIR32
+ LIBDIR32 ":"
+#endif
+#ifdef LIBDIR64
+ LIBDIR64 ":"
+#endif
+ "%s", env ? env : "");
+ } else {
+ /* [$PATH/LIB32:][$PATH/LIB64:]$LD_LIBRARY_PATH */
+ snprintf(val, sizeof(val), ""
+#ifdef LIBDIR32
+ "%s/" LIB32 ":"
+#endif
+#ifdef LIBDIR64
+ "%s/" LIB64 ":"
+#endif
+ "%s",
+#ifdef LIBDIR32
+ path,
+#endif
+#ifdef LIBDIR64
+ path,
+#endif
+ env ? env : "");
+ }
+ setup_environment("LD_LIBRARY_PATH", val);
+}
+
void ldpreload(int which)
{
if (which == MAP_HEAP) {
@@ -154,12 +214,16 @@ int main(int argc, char** argv)
{
int opt_mappings = 0;
int opt_preload = 1;
+ char *opt_library = NULL;
char opts[] = "+h";
int ret = 0, index = 0;
struct option long_opts[] = {
{"help", no_argument, NULL, 'h'},
{"no-preload", no_argument, NULL, LONG_NO_PRELOAD},
+ {"library-use-path",
+ no_argument, NULL, LONG_NO_LIBRARY},
+ {"library",required_argument, NULL, LONG_LIBRARY},
{"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
{"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
@@ -192,6 +256,15 @@ int main(int argc, char** argv)
DEBUG("LD_PRELOAD disabled\n");
break;
+ case LONG_NO_LIBRARY:
+ opt_library = LIBRARY_DISABLE;
+ DEBUG("using LD_LIBRARY_PATH to find library\n");
+ break;
+
+ case LONG_LIBRARY:
+ opt_library = optarg;
+ break;
+
default:
WARNING("unparsed option %08x\n", ret);
ret = -1;
@@ -206,6 +279,9 @@ int main(in
Re: [Libhugetlbfs-devel] [PATCH 2/6] hugectl: add remap support via --text, --data, --bss, and --disable
On (24/07/08 17:44), Andy Whitcroft didst pronounce:
> Add control over remap via the --text, --data, --bss, and --disable
> options. The first three request mapping of those segments, the last
> disables all remap. Where the combinations requested cannot be exactly
> handled then the request is "widened" to get that segment remapped,
> for example if you request --data or --bss in isolation then both are
> remapped and a warning emitted.
>
> Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
I'm happy with the interface at least
Acked-by: Mel Gorman <[EMAIL PROTECTED]>
> ---
> hugectl.c | 71
> +
> 1 files changed, 71 insertions(+), 0 deletions(-)
>
> diff --git a/hugectl.c b/hugectl.c
> index ffbd260..0387c5b 100644
> --- a/hugectl.c
> +++ b/hugectl.c
> @@ -58,6 +58,10 @@ void print_usage()
> fprintf(stderr, "options:\n");
>
> OPTION("--help, -h", "Prints this message");
> +
> + OPTION("--text", "Requests remapping of the program text");
> + OPTION("--data", "Requests remapping of the program data");
> + OPTION("--bss", "Requests remapping of the program bss");
> }
>
> int verbose_level = VERBOSITY_DEFAULT;
> @@ -74,12 +78,71 @@ void verbose_init(void)
> verbose_level = VERBOSITY_MAX;
> }
>
> +void setup_environment(char *var, char *val)
> +{
> + setenv(var, val, 1);
> + DEBUG("%s='%s'\n", var, val);
> +}
> +
> +
> +/*
> + * getopts return values for options which are long only.
> + */
> +#define MAP_BASE 0x1000
> +
> +/*
> + * Mapping selectors, one bit per remappable/backable area as requested
> + * by the user. These are also used as returns from getopts where they
> + * are offset from MAP_BASE, which must be removed before they are compared.
> + */
> +#define MAP_DISABLE 0x0001
> +#define MAP_TEXT 0x0002
> +#define MAP_DATA 0x0004
> +#define MAP_BSS 0x0008
> +
> +void setup_mappings(int which)
> +{
> + char remap[3] = { 0, 0, 0 };
> + int n = 0;
> +
> + /*
> + * HUGETLB_ELFMAP should be set to either a combination of 'R' and 'W'
> + * which indicate which segments should be remapped. It may also be
> + * set to 'no' to prevent remapping.
> + */
> + if (which & MAP_TEXT)
> + remap[n++] = 'R';
> + if (which & (MAP_DATA|MAP_BSS)) {
> + if ((which & (MAP_DATA|MAP_BSS)) != (MAP_DATA|MAP_BSS))
> + WARNING("data and bss remapped together\n");
> + remap[n++] = 'W';
> + }
> + if (which & MAP_DISABLE) {
> + if (which != MAP_DISABLE)
> + WARNING("--disable masks requested remap\n");
> + n = 0;
> + remap[n++] = 'n';
> + remap[n++] = 'o';
> + }
> +
> + if (n)
> + setup_environment("HUGETLB_ELFMAP", remap);
> +}
> +
> int main(int argc, char** argv)
> {
> + int opt_mappings = 0;
> +
> char opts[] = "+h";
> int ret = 0, index = 0;
> struct option long_opts[] = {
> {"help", no_argument, NULL, 'h'},
> +
> + {"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
> + {"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
> + {"data", no_argument, NULL, MAP_BASE|MAP_DATA},
> + {"bss",no_argument, NULL, MAP_BASE|MAP_BSS},
> +
> {0},
> };
>
> @@ -87,6 +150,10 @@ int main(int argc, char** argv)
>
> while (ret != -1) {
> ret = getopt_long(argc, argv, opts, long_opts, &index);
> + if (ret > 0 && (ret & MAP_BASE)) {
> + opt_mappings |= ret;
> + continue;
> + }
> switch (ret) {
> case '?':
> print_usage();
> @@ -103,12 +170,16 @@ int main(int argc, char** argv)
> }
> }
> index = optind;
> + opt_mappings &= ~MAP_BASE;
>
> if ((argc - index) < 1) {
> print_usage();
> exit(EXIT_FAILURE);
> }
>
> + if (opt_mappings)
> + setup_mappings(opt_mappings);
> +
> execvp(argv[index], &argv[index]);
> ERROR("exec failed: %s\n", strerror(errno));
> exit(EXIT_FAILURE);
> --
> 1.5.6.GIT
>
--
Mel Gorman
Part-time Phd Student Linux Technology Center
University of Limerick IBM Dublin Software Lab
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.so
Re: [Libhugetlbfs-devel] [PATCH 1/2] build: Cleanup tools build V2
On 24.07.2008 [15:45:01 +], Adam Litke wrote: > To properly support building and installing tools (hugectl, hugeedit, hugecfg, > etc), some changes to the initial method of building hugectl are required. > This patch creates wildcard rules that can be used for building either 32 or > 64 > bit versions of all tools. Additionally, a native wordsize is selected for > each architecture so that only one binary (of the desired default word size) > is > built and installed. > > Changes since V1: > - Allow the compiler to choose whether to build 32 or 64 bit tools > > Signed-off-by: Adam Litke <[EMAIL PROTECTED]> Applied 1/2 and 2/2, thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 2/2] tools: hugeedit: Modify binaries to set default remapping behavior V4
The recently added relinking method makes it possible to choose from multiple
ways to remap segments at run-time via the HUGTLB_ELFMAP environment variable.
If nothing is specified in that variable, then no remapping will occur.
Sometimes it is desirable to set a default remapping mode so that
HUGETLB_ELFMAP does not need to be set. This semantic applied to the original
remapping algorithm.
This patch adds a utility that can modify the PF_LINUX_HUGETLB segment flags in
the ELF haeder of a binary to control which segments will be remapped by
default. The tool will also display this information.
Changes since V3 (Thanks Andy)
- Use REPORT infrastructure for error messages
- --disable cannot be specified with --text and/or --data
- Usage message macros
- Check return values of munmap and close to avoid silent data loss
- Whitespace cleanups
Changes since V2 (Thanks again Mel)
- Support for remapping more of the ELF header if one page is not enough
- is_text() checks that p_filesz == p_memsz in addition to permissions
- Rename scan_phdrs* to update_phdrs*
- Print segment start and end addresses
- Whitespace
Changes since V1 (Thanks Mel for the review):
- New --text, --data, and --disable options
- Binary default modes are always displayed
- Binary is opened for writing only when updates have been requested
Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
---
Makefile|2
hugeedit.c | 239 +++
libhugetlbfs_internal.h |2
3 files changed, 242 insertions(+), 1 deletions(-)
create mode 100644 hugeedit.c
diff --git a/Makefile b/Makefile
index 8030d30..f1f83fa 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ EXEDIR = /bin
LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o
INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
BIN_OBJ_DIR=obj
-INSTALL_BIN = hugectl
+INSTALL_BIN = hugectl hugeedit
INSTALL_HEADERS = hugetlbfs.h
LDSCRIPT_TYPES = B BDT
LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64
diff --git a/hugeedit.c b/hugeedit.c
new file mode 100644
index 000..3bfcfe3
--- /dev/null
+++ b/hugeedit.c
@@ -0,0 +1,239 @@
+/*
+ * libhugetlbfs - Easy use of Linux hugepages
+ * Copyright (C) 2008 Adam Litke, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public License
+ * as published by the Free Software Foundation; either version 2.1 of
+ * the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#define _GNU_SOURCE
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+/*
+ * Eventually we plan to use the libhugetlbfs reporting facility,
+ * but until that is possible, redefine a simpler version here.
+ */
+#define REPORT(level, prefix, format, ...) \
+ do { \
+ fprintf(stderr, "hugeedit: " prefix ": " format, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#include "libhugetlbfs_internal.h"
+
+/*
+ * All MAP_* options are tagged with MAP_BASE to differentiate them as options
+ * in the options parser. This must be removed before they are compared.
+ */
+#define MAP_BASE 0x1000
+#define MAP_DISABLE0x0001
+#define MAP_TEXT 0x0002
+#define MAP_DATA 0x0004
+
+#define PF_LINUX_HUGETLB 0x10
+extern int optind;
+extern char *optarg;
+
+#define OPTION(opts, text) fprintf(stderr, " %-25s %s\n", opts, text)
+#define CONT(text) fprintf(stderr, " %-25s %s\n", "", text)
+
+void print_usage()
+{
+ printf("hugeedit [options] target\n");
+ printf("options:\n");
+ OPTION("--text", "Remap program text into huge pages by default");
+ OPTION("--data", "Remap program data into huge pages by default");
+ OPTION("--disable", "Remap no segments into huge pages by default");
+ OPTION("--help", "Print this usage information");
+}
+
+int check_elf_wordsize(void *ehdr)
+{
+ char *e_ident = (char *) ehdr;
+
+ if (strncmp(e_ident, "\177ELF", 4)) {
+ ERROR("Not a valid ELF executable\n");
+ exit(EXIT_FAILURE);
+ }
+
+ switch (e_ident[EI_CLASS]) {
+ case ELFCLASS32:
+ case ELFCLASS64:
+ return e_ident[EI_CLASS];
+ default:
+ ERROR("Can not determine word size\n");
+ exit(EXIT_FAILURE);
Re: [Libhugetlbfs-devel] [PATCH 0/3] [V1] hugectl cleanups II
On 24.07.2008 [14:46:17 +0100], Andy Whitcroft wrote: > Following this email are three further cleanups for the basic hugectl > infrastructure, and its interface to the library debugging. These > have come out of the ongoing work on hugectl, but stand alone as > cleanups. > > Following this email are three patches: > > debug: expose default and maximum verbosity level -- which exposes the > library default and maxmimum verbosity levels so that other applications > can track the same levels, > > hugectl: convert exit status to POSIX standard codes -- which fixes up > the exit statuses to be POSIX compliant, and > > hugectl: simplify per-option help text -- which simplifies specification > of the help text for new options. > > These should be reasonably obvious and clean, please consider for > application. Applied, thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 1/2] build: Cleanup tools build V2
On 24.07.2008 [09:59:37 -0700], Nishanth Aravamudan wrote: > On 24.07.2008 [15:45:01 +], Adam Litke wrote: > > To properly support building and installing tools (hugectl, hugeedit, > > hugecfg, > > etc), some changes to the initial method of building hugectl are required. > > This patch creates wildcard rules that can be used for building either 32 > > or 64 > > bit versions of all tools. Additionally, a native wordsize is selected for > > each architecture so that only one binary (of the desired default word > > size) is > > built and installed. > > > > Changes since V1: > > - Allow the compiler to choose whether to build 32 or 64 bit tools > > > > Signed-off-by: Adam Litke <[EMAIL PROTECTED]> > > Applied 1/2 and 2/2, thanks, I fib, I've waited on 2/2 until you've had a chance to reply/fix Andy's nits. 1/2 is applied, though. Thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH] tools: hugeedit: Modify binaries to set default remapping behavior V5
The recently added relinking method makes it possible to choose from multiple
ways to remap segments at run-time via the HUGTLB_ELFMAP environment variable.
If nothing is specified in that variable, then no remapping will occur.
Sometimes it is desirable to set a default remapping mode so that
HUGETLB_ELFMAP does not need to be set. This semantic applied to the original
remapping algorithm.
This patch adds a utility that can modify the PF_LINUX_HUGETLB segment flags in
the ELF haeder of a binary to control which segments will be remapped by
default. The tool will also display this information.
Changes since V4
- getopt fixes and a few cosmetic changes
Changes since V3 (Thanks Andy)
- Use REPORT infrastructure for error messages
- --disable cannot be specified with --text and/or --data
- Usage message macros
- Check return values of munmap and close to avoid silent data loss
- Whitespace cleanups
Changes since V2 (Thanks again Mel)
- Support for remapping more of the ELF header if one page is not enough
- is_text() checks that p_filesz == p_memsz in addition to permissions
- Rename scan_phdrs* to update_phdrs*
- Print segment start and end addresses
- Whitespace
Changes since V1 (Thanks Mel for the review):
- New --text, --data, and --disable options
- Binary default modes are always displayed
- Binary is opened for writing only when updates have been requested
Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
---
Makefile|2
hugeedit.c | 240 +++
libhugetlbfs_internal.h |2
3 files changed, 243 insertions(+), 1 deletions(-)
create mode 100644 hugeedit.c
diff --git a/Makefile b/Makefile
index 8030d30..f1f83fa 100644
--- a/Makefile
+++ b/Makefile
@@ -4,7 +4,7 @@ EXEDIR = /bin
LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o
INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
BIN_OBJ_DIR=obj
-INSTALL_BIN = hugectl
+INSTALL_BIN = hugectl hugeedit
INSTALL_HEADERS = hugetlbfs.h
LDSCRIPT_TYPES = B BDT
LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64
diff --git a/hugeedit.c b/hugeedit.c
new file mode 100644
index 000..eea0e51
--- /dev/null
+++ b/hugeedit.c
@@ -0,0 +1,240 @@
+/*
+ * libhugetlbfs - Easy use of Linux hugepages
+ * Copyright (C) 2008 Adam Litke, IBM Corporation.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public License
+ * as published by the Free Software Foundation; either version 2.1 of
+ * the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#define _GNU_SOURCE
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+/*
+ * Eventually we plan to use the libhugetlbfs reporting facility,
+ * but until that is possible, redefine a simpler version here.
+ */
+#define REPORT(level, prefix, format, ...) \
+ do { \
+ fprintf(stderr, "hugeedit: " prefix ": " format, \
+ ##__VA_ARGS__); \
+ } while (0)
+
+#include "libhugetlbfs_internal.h"
+
+/*
+ * All MAP_* options are tagged with MAP_BASE to differentiate them as options
+ * in the options parser. This must be removed before they are compared.
+ */
+#define MAP_BASE 0x1000
+#define MAP_DISABLE0x0001
+#define MAP_TEXT 0x0002
+#define MAP_DATA 0x0004
+
+#define PF_LINUX_HUGETLB 0x10
+extern int optind;
+extern char *optarg;
+
+#define OPTION(opts, text) fprintf(stderr, " %-25s %s\n", opts, text)
+#define CONT(text) fprintf(stderr, " %-25s %s\n", "", text)
+
+void print_usage()
+{
+ printf("hugeedit [options] target\n");
+ printf("options:\n");
+ OPTION("--text", "Remap program text into huge pages by default");
+ OPTION("--data", "Remap program data into huge pages by default");
+ OPTION("--disable", "Remap no segments into huge pages by default");
+ OPTION("--help, -h", "Print this usage information");
+}
+
+int check_elf_wordsize(void *ehdr)
+{
+ char *e_ident = (char *) ehdr;
+
+ if (strncmp(e_ident, ELFMAG, SELFMAG)) {
+ ERROR("Not a valid ELF executable\n");
+ exit(EXIT_FAILURE);
+ }
+
+ switch (e_ident[EI_CLASS]) {
+ case ELFCLASS32:
+ case ELFCLASS64:
+ return e_ident[EI_CLASS];
+ default:
+ ERROR("Can not d
Re: [Libhugetlbfs-devel] [PATCH 2/6] hugectl: add remap support via --text, --data, --bss, and --disable
On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote:
> Add control over remap via the --text, --data, --bss, and --disable
> options. The first three request mapping of those segments, the last
> disables all remap. Where the combinations requested cannot be exactly
> handled then the request is "widened" to get that segment remapped,
> for example if you request --data or --bss in isolation then both are
> remapped and a warning emitted.
>
> Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
Seems like this has been hashed out sufficiently and I am happy with the
result.
Acked-by: Adam Litke <[EMAIL PROTECTED]>
> ---
> hugectl.c | 71
> +
> 1 files changed, 71 insertions(+), 0 deletions(-)
>
> diff --git a/hugectl.c b/hugectl.c
> index ffbd260..0387c5b 100644
> --- a/hugectl.c
> +++ b/hugectl.c
> @@ -58,6 +58,10 @@ void print_usage()
> fprintf(stderr, "options:\n");
>
> OPTION("--help, -h", "Prints this message");
> +
> + OPTION("--text", "Requests remapping of the program text");
> + OPTION("--data", "Requests remapping of the program data");
> + OPTION("--bss", "Requests remapping of the program bss");
> }
>
> int verbose_level = VERBOSITY_DEFAULT;
> @@ -74,12 +78,71 @@ void verbose_init(void)
> verbose_level = VERBOSITY_MAX;
> }
>
> +void setup_environment(char *var, char *val)
> +{
> + setenv(var, val, 1);
> + DEBUG("%s='%s'\n", var, val);
> +}
> +
> +
> +/*
> + * getopts return values for options which are long only.
> + */
> +#define MAP_BASE 0x1000
> +
> +/*
> + * Mapping selectors, one bit per remappable/backable area as requested
> + * by the user. These are also used as returns from getopts where they
> + * are offset from MAP_BASE, which must be removed before they are compared.
> + */
> +#define MAP_DISABLE 0x0001
> +#define MAP_TEXT 0x0002
> +#define MAP_DATA 0x0004
> +#define MAP_BSS 0x0008
> +
> +void setup_mappings(int which)
> +{
> + char remap[3] = { 0, 0, 0 };
> + int n = 0;
> +
> + /*
> + * HUGETLB_ELFMAP should be set to either a combination of 'R' and 'W'
> + * which indicate which segments should be remapped. It may also be
> + * set to 'no' to prevent remapping.
> + */
> + if (which & MAP_TEXT)
> + remap[n++] = 'R';
> + if (which & (MAP_DATA|MAP_BSS)) {
> + if ((which & (MAP_DATA|MAP_BSS)) != (MAP_DATA|MAP_BSS))
> + WARNING("data and bss remapped together\n");
> + remap[n++] = 'W';
> + }
> + if (which & MAP_DISABLE) {
> + if (which != MAP_DISABLE)
> + WARNING("--disable masks requested remap\n");
> + n = 0;
> + remap[n++] = 'n';
> + remap[n++] = 'o';
> + }
> +
> + if (n)
> + setup_environment("HUGETLB_ELFMAP", remap);
> +}
> +
> int main(int argc, char** argv)
> {
> + int opt_mappings = 0;
> +
> char opts[] = "+h";
> int ret = 0, index = 0;
> struct option long_opts[] = {
> {"help", no_argument, NULL, 'h'},
> +
> + {"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
> + {"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
> + {"data", no_argument, NULL, MAP_BASE|MAP_DATA},
> + {"bss",no_argument, NULL, MAP_BASE|MAP_BSS},
> +
> {0},
> };
>
> @@ -87,6 +150,10 @@ int main(int argc, char** argv)
>
> while (ret != -1) {
> ret = getopt_long(argc, argv, opts, long_opts, &index);
> + if (ret > 0 && (ret & MAP_BASE)) {
> + opt_mappings |= ret;
> + continue;
> + }
> switch (ret) {
> case '?':
> print_usage();
> @@ -103,12 +170,16 @@ int main(int argc, char** argv)
> }
> }
> index = optind;
> + opt_mappings &= ~MAP_BASE;
>
> if ((argc - index) < 1) {
> print_usage();
> exit(EXIT_FAILURE);
> }
>
> + if (opt_mappings)
> + setup_mappings(opt_mappings);
> +
> execvp(argv[index], &argv[index]);
> ERROR("exec failed: %s\n", strerror(errno));
> exit(EXIT_FAILURE);
--
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 3/6] hugectl: add backing of the heap via --heap
On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote:
> Add support for mapping the heap via morecore through the --heap option.
>
> Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
> Acked-by: Mel Gorman <[EMAIL PROTECTED]>
Acked-by: Adam Litke <[EMAIL PROTECTED]>
> ---
> hugectl.c |7 +++
> 1 files changed, 7 insertions(+), 0 deletions(-)
>
> diff --git a/hugectl.c b/hugectl.c
> index 0387c5b..79e934c 100644
> --- a/hugectl.c
> +++ b/hugectl.c
> @@ -62,6 +62,8 @@ void print_usage()
> OPTION("--text", "Requests remapping of the program text");
> OPTION("--data", "Requests remapping of the program data");
> OPTION("--bss", "Requests remapping of the program bss");
> + OPTION("--heap", "Requests remapping of the program heap");
> + CONT("(malloc space)");
> }
>
> int verbose_level = VERBOSITY_DEFAULT;
> @@ -99,6 +101,7 @@ void setup_environment(char *var, char *val)
> #define MAP_TEXT 0x0002
> #define MAP_DATA 0x0004
> #define MAP_BSS 0x0008
> +#define MAP_HEAP 0x0010
>
> void setup_mappings(int which)
> {
> @@ -127,6 +130,9 @@ void setup_mappings(int which)
>
> if (n)
> setup_environment("HUGETLB_ELFMAP", remap);
> +
> + if (which & MAP_HEAP)
> + setup_environment("HUGETLB_MORECORE", "yes");
> }
>
> int main(int argc, char** argv)
> @@ -142,6 +148,7 @@ int main(int argc, char** argv)
> {"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
> {"data", no_argument, NULL, MAP_BASE|MAP_DATA},
> {"bss",no_argument, NULL, MAP_BASE|MAP_BSS},
> + {"heap", no_argument, NULL, MAP_BASE|MAP_HEAP},
>
> {0},
> };
--
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 4/6] hugectl: add support for preload
On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote:
> Preload is only useful when linking against an non-relinked binary
> and only then when we are only attempting to back the heap. No other
> combination can usefully use preload. Using preload with a relinked
> binary is safe as long as you are only requesting backing of the heap.
> Therefore unconditionally enable preload iff we are requesting only --heap.
> As a failsafe allow the user to disable preload.
>
> Note that we make use of the fact that for a standard install the
> libraries will appear on the default libarary path, and that the loader
> will correctly pick an appropriate library 32/64 bit based on the binary.
>
> Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
> Acked-by: Mel Gorman <[EMAIL PROTECTED]>
Seems like the decision to always preload with a lone --heap is
reasonable. If we run into trouble down the road then we can employ a
more sophisticated heuristic to determine when to preload.
Acked-by: Adam Litke <[EMAIL PROTECTED]>
> ---
> hugectl.c | 25 +
> 1 files changed, 25 insertions(+), 0 deletions(-)
>
> diff --git a/hugectl.c b/hugectl.c
> index 79e934c..755d3b6 100644
> --- a/hugectl.c
> +++ b/hugectl.c
> @@ -64,6 +64,8 @@ void print_usage()
> OPTION("--bss", "Requests remapping of the program bss");
> OPTION("--heap", "Requests remapping of the program heap");
> CONT("(malloc space)");
> +
> + OPTION("--no-preload", "Disable preloading the libhugetlbfs library");
> }
>
> int verbose_level = VERBOSITY_DEFAULT;
> @@ -91,6 +93,9 @@ void setup_environment(char *var, char *val)
> * getopts return values for options which are long only.
> */
> #define MAP_BASE 0x1000
> +#define LONG_BASE0x2000
> +
> +#define LONG_NO_PRELOAD (LONG_BASE | 'p')
>
> /*
> * Mapping selectors, one bit per remappable/backable area as requested
> @@ -135,14 +140,26 @@ void setup_mappings(int which)
> setup_environment("HUGETLB_MORECORE", "yes");
> }
>
> +void ldpreload(int which)
> +{
> + if (which == MAP_HEAP) {
> + setup_environment("LD_PRELOAD", "libhugetlbfs.so");
> + WARNING("LD_PRELOAD in use for lone --heap\n");
> + } else {
> + DEBUG("LD_PRELOAD not appropriate for this map combination\n");
> + }
> +}
> +
> int main(int argc, char** argv)
> {
> int opt_mappings = 0;
> + int opt_preload = 1;
>
> char opts[] = "+h";
> int ret = 0, index = 0;
> struct option long_opts[] = {
> {"help", no_argument, NULL, 'h'},
> + {"no-preload", no_argument, NULL, LONG_NO_PRELOAD},
>
> {"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
> {"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
> @@ -170,6 +187,11 @@ int main(int argc, char** argv)
> print_usage();
> exit(EXIT_SUCCESS);
>
> + case LONG_NO_PRELOAD:
> + opt_preload = 0;
> + DEBUG("LD_PRELOAD disabled\n");
> + break;
> +
> default:
> WARNING("unparsed option %08x\n", ret);
> ret = -1;
> @@ -187,6 +209,9 @@ int main(int argc, char** argv)
> if (opt_mappings)
> setup_mappings(opt_mappings);
>
> + if (opt_preload)
> + ldpreload(opt_mappings);
> +
> execvp(argv[index], &argv[index]);
> ERROR("exec failed: %s\n", strerror(errno));
> exit(EXIT_FAILURE);
--
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH] tools: hugeedit: Modify binaries to set default remapping behavior V5
On Thu, Jul 24, 2008 at 06:12:02PM +, Adam Litke wrote:
> The recently added relinking method makes it possible to choose from multiple
> ways to remap segments at run-time via the HUGTLB_ELFMAP environment variable.
> If nothing is specified in that variable, then no remapping will occur.
> Sometimes it is desirable to set a default remapping mode so that
> HUGETLB_ELFMAP does not need to be set. This semantic applied to the original
> remapping algorithm.
>
> This patch adds a utility that can modify the PF_LINUX_HUGETLB segment flags
> in
> the ELF haeder of a binary to control which segments will be remapped by
> default. The tool will also display this information.
>
> Changes since V4
> - getopt fixes and a few cosmetic changes
>
> Changes since V3 (Thanks Andy)
> - Use REPORT infrastructure for error messages
> - --disable cannot be specified with --text and/or --data
> - Usage message macros
> - Check return values of munmap and close to avoid silent data loss
> - Whitespace cleanups
>
> Changes since V2 (Thanks again Mel)
> - Support for remapping more of the ELF header if one page is not enough
> - is_text() checks that p_filesz == p_memsz in addition to permissions
> - Rename scan_phdrs* to update_phdrs*
> - Print segment start and end addresses
> - Whitespace
>
> Changes since V1 (Thanks Mel for the review):
> - New --text, --data, and --disable options
> - Binary default modes are always displayed
> - Binary is opened for writing only when updates have been requested
>
> Signed-off-by: Adam Litke <[EMAIL PROTECTED]>
> ---
>
> Makefile|2
> hugeedit.c | 240
> +++
> libhugetlbfs_internal.h |2
> 3 files changed, 243 insertions(+), 1 deletions(-)
> create mode 100644 hugeedit.c
>
>
> diff --git a/Makefile b/Makefile
> index 8030d30..f1f83fa 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -4,7 +4,7 @@ EXEDIR = /bin
> LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o
> INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
> BIN_OBJ_DIR=obj
> -INSTALL_BIN = hugectl
> +INSTALL_BIN = hugectl hugeedit
> INSTALL_HEADERS = hugetlbfs.h
> LDSCRIPT_TYPES = B BDT
> LDSCRIPT_DIST_ELF = elf32ppclinux elf64ppc elf_i386 elf_x86_64
> diff --git a/hugeedit.c b/hugeedit.c
> new file mode 100644
> index 000..eea0e51
> --- /dev/null
> +++ b/hugeedit.c
> @@ -0,0 +1,240 @@
> +/*
> + * libhugetlbfs - Easy use of Linux hugepages
> + * Copyright (C) 2008 Adam Litke, IBM Corporation.
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public License
> + * as published by the Free Software Foundation; either version 2.1 of
> + * the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#define _GNU_SOURCE
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +
> +/*
> + * Eventually we plan to use the libhugetlbfs reporting facility,
> + * but until that is possible, redefine a simpler version here.
> + */
> +#define REPORT(level, prefix, format, ...) \
> + do { \
> + fprintf(stderr, "hugeedit: " prefix ": " format, \
> + ##__VA_ARGS__); \
> + } while (0)
> +
> +#include "libhugetlbfs_internal.h"
> +
> +/*
> + * All MAP_* options are tagged with MAP_BASE to differentiate them as
> options
> + * in the options parser. This must be removed before they are compared.
> + */
> +#define MAP_BASE 0x1000
> +#define MAP_DISABLE 0x0001
> +#define MAP_TEXT 0x0002
> +#define MAP_DATA 0x0004
> +
> +#define PF_LINUX_HUGETLB 0x10
> +extern int optind;
> +extern char *optarg;
> +
> +#define OPTION(opts, text) fprintf(stderr, " %-25s %s\n", opts, text)
> +#define CONT(text) fprintf(stderr, " %-25s %s\n", "", text)
> +
> +void print_usage()
> +{
> + printf("hugeedit [options] target\n");
> + printf("options:\n");
> + OPTION("--text", "Remap program text into huge pages by default");
> + OPTION("--data", "Remap program data into huge pages by default");
> + OPTION("--disable", "Remap no segments into huge pages by default");
> + OPTION("--help, -h", "Print this usage information");
> +}
> +
> +int check_elf_wordsize(void *ehdr)
> +{
> + char *e_ident = (char *) ehdr;
> +
> + if (strncmp(e_ident, ELFMAG, SELFMAG)) {
> +
Re: [Libhugetlbfs-devel] [PATCH 6/6] hugectl: add a --dry-run option to dump what would be done
On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote:
> Allow the user to request the specific hugetlb configuration which will
> be used for the selected options, exposed via the --dry-run option.
>
> Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
Acked-by: Adam Litke <[EMAIL PROTECTED]>
> ---
> hugectl.c | 18 +-
> 1 files changed, 17 insertions(+), 1 deletions(-)
>
> diff --git a/hugectl.c b/hugectl.c
> index 81e76b2..f369908 100644
> --- a/hugectl.c
> +++ b/hugectl.c
> @@ -78,8 +78,11 @@ void print_usage()
> LIBDIR32 ":"
> #endif
> ")");
> +
> + OPTION("--dry-run", "describe what would be done without doing it");
> }
>
> +int opt_dry_run = 0;
> int verbose_level = VERBOSITY_DEFAULT;
>
> void verbose_init(void)
> @@ -98,6 +101,9 @@ void setup_environment(char *var, char *val)
> {
> setenv(var, val, 1);
> DEBUG("%s='%s'\n", var, val);
> +
> + if (opt_dry_run)
> + printf("%s='%s'\n", var, val);
> }
>
>
> @@ -112,6 +118,8 @@ void setup_environment(char *var, char *val)
> #define LONG_NO_LIBRARY (LONG_BASE | 'L')
> #define LONG_LIBRARY (LONG_BASE | 'l')
>
> +#define LONG_DRY_RUN (LONG_BASE | 'd')
> +
> /*
> * Mapping selectors, one bit per remappable/backable area as requested
> * by the user. These are also used as returns from getopts where they
> @@ -224,6 +232,7 @@ int main(int argc, char** argv)
> {"library-use-path",
> no_argument, NULL, LONG_NO_LIBRARY},
> {"library",required_argument, NULL, LONG_LIBRARY},
> + {"dry-run",no_argument, NULL, LONG_DRY_RUN},
>
> {"disable",no_argument, NULL, MAP_BASE|MAP_DISABLE},
> {"text", no_argument, NULL, MAP_BASE|MAP_TEXT},
> @@ -265,6 +274,10 @@ int main(int argc, char** argv)
> opt_library = optarg;
> break;
>
> + case LONG_DRY_RUN:
> + opt_dry_run = 1;
> + break;
> +
> default:
> WARNING("unparsed option %08x\n", ret);
> ret = -1;
> @@ -274,7 +287,7 @@ int main(int argc, char** argv)
> index = optind;
> opt_mappings &= ~MAP_BASE;
>
> - if ((argc - index) < 1) {
> + if (!opt_dry_run && (argc - index) < 1) {
> print_usage();
> exit(EXIT_FAILURE);
> }
> @@ -288,6 +301,9 @@ int main(int argc, char** argv)
> if (opt_preload)
> ldpreload(opt_mappings);
>
> + if (opt_dry_run)
> + exit(EXIT_SUCCESS);
> +
> execvp(argv[index], &argv[index]);
> ERROR("exec failed: %s\n", strerror(errno));
> exit(EXIT_FAILURE);
--
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 5/6] hugectl: add support for requesting which library to use
On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote:
> Add support for requesting a specific library set for preload. This adds
> the --library option which takes an optional argument. Without an argument
> it requests use of the specific libraries installed with the version
> of hugectl in use. An argument is treated as a prefix to the library
> directories and the approprate 32/64 bit library directories are added.
Hmm, there is a use case that this patch doesn't cover (and it was the
first thing I thought of when I saw the patch title). Say I have a
pre-built library in /home/aglitke (or some such path without lib/lib64
structure) that I want to use. The patch (as it exists now) would not
let me point hugectl at this library. Am I the only one who thinks this
might be useful? I suppose one option is to make users wanting to do
this simply use LD_LIBRARY_PATH directly.
> Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
> ---
> Makefile |9 ++-
> hugectl.c | 76
> +
> 2 files changed, 84 insertions(+), 1 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index d69a365..6571672 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -101,6 +101,13 @@ BINDIR = $(PREFIX)/share/libhugetlbfs
> EXEDIR = $(PREFIX)/bin
> DOCDIR = $(PREFIX)/share/doc/libhugetlbfs
>
> +ifdef LIB32
> +LIBPATHS += -DLIB32='"$(LIB32)"' -DLIBDIR32='"$(LIBDIR32)"'
> +endif
> +ifdef LIB64
> +LIBPATHS += -DLIB64='"$(LIB64)"' -DLIBDIR64='"$(LIBDIR64)"'
> +endif
> +
> EXTRA_DIST = \
> README \
> HOWTO \
> @@ -228,7 +235,7 @@ obj64/%.s:%.c
>
> $(OBJS): hugectl.c
> @$(VECHO) CPP $@
> - $(CC) $(CFLAGS) -o $@ -c $<
> + $(CC) $(CFLAGS) $(LIBPATHS) -o $@ -c $<
>
> $(INSTALL_OBJ): $(OBJS)
> @$(VECHO) CC $@
> diff --git a/hugectl.c b/hugectl.c
> index 755d3b6..81e76b2 100644
> --- a/hugectl.c
> +++ b/hugectl.c
> @@ -31,6 +31,7 @@
> #include
> #include
> #include
> +#include
>
> #define _GNU_SOURCE /* for getopt_long */
> #include
> @@ -66,6 +67,17 @@ void print_usage()
> CONT("(malloc space)");
>
> OPTION("--no-preload", "Disable preloading the libhugetlbfs library");
> +
> + OPTION("--library-use-path", "Use the system library path");
> + OPTION("--library ", "Select a library prefix");
> + CONT("(Default: "
> +#ifdef LIBDIR32
> + LIBDIR32 ":"
> +#endif
> +#ifdef LIBDIR32
> + LIBDIR32 ":"
> +#endif
> + ")");
> }
>
> int verbose_level = VERBOSITY_DEFAULT;
> @@ -97,6 +109,9 @@ void setup_environment(char *var, char *val)
>
> #define LONG_NO_PRELOAD (LONG_BASE | 'p')
>
> +#define LONG_NO_LIBRARY (LONG_BASE | 'L')
> +#define LONG_LIBRARY (LONG_BASE | 'l')
> +
> /*
> * Mapping selectors, one bit per remappable/backable area as requested
> * by the user. These are also used as returns from getopts where they
> @@ -140,6 +155,51 @@ void setup_mappings(int which)
> setup_environment("HUGETLB_MORECORE", "yes");
> }
>
> +#define LIBRARY_DISABLE ((void *)-1)
> +
> +void library_path(char *path)
> +{
> + char val[NAME_MAX] = "";
> + char *env;
> +
> + /*
> + * Select which libraries we wish to use. No argument implies
> + * use the libraries included with this version of hugectl.
> + * Else it is assumed to be a prefix to the 32/64 bit directories
> + * both of which are added, where available.
> + */
> + env = getenv("LD_LIBRARY_PATH");
> + if (!path) {
> + /* [LIBDIR32:][LIBDIR64:]$LD_LIBRARY_PATH */
> + snprintf(val, sizeof(val), ""
> +#ifdef LIBDIR32
> + LIBDIR32 ":"
> +#endif
> +#ifdef LIBDIR64
> + LIBDIR64 ":"
> +#endif
> + "%s", env ? env : "");
> + } else {
> + /* [$PATH/LIB32:][$PATH/LIB64:]$LD_LIBRARY_PATH */
> + snprintf(val, sizeof(val), ""
> +#ifdef LIBDIR32
> + "%s/" LIB32 ":"
> +#endif
> +#ifdef LIBDIR64
> + "%s/" LIB64 ":"
> +#endif
> + "%s",
> +#ifdef LIBDIR32
> + path,
> +#endif
> +#ifdef LIBDIR64
> + path,
> +#endif
> + env ? env : "");
> + }
> + setup_environment("LD_LIBRARY_PATH", val);
> +}
> +
> void ldpreload(int which)
> {
> if (which == MAP_HEAP) {
> @@ -154,12 +214,16 @@ int main(int argc, char** argv)
> {
> int opt_mappings = 0;
> int opt_preload = 1;
> + char *opt_library = NULL;
>
> char opts[] = "+h";
> int ret = 0, index = 0;
> struct option long_opts[] = {
> {"help", no_argument, NULL, 'h'},
> {"no-preload", no_argument, NULL, LONG_NO_PRELOAD},
> + {"library-use-path",
> +no_argument, NULL, LONG_NO_LIBRARY},
> + {"library",required_argument, NULL, LONG_LIBRARY},
>
> {"disable",no_argument, NULL, MAP_B
Re: [Libhugetlbfs-devel] [PATCH 1/6] hugectl: honour the hugetlb verbosity/debug flags
On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote:
> Handle the HUGETLB_VERBOSE and HUGETLB_DEBUG flags such that we will emit
> messages at the same levels as defined for the library. Use the common
> header to define the levels. Convert all existing errors to use the new
> macros.
>
> Signed-off-by: Andy Whitcroft <[EMAIL PROTECTED]>
> Acked-by: Mel Gorman <[EMAIL PROTECTED]>
Acked-by: Adam Litke <[EMAIL PROTECTED]>
> ---
> hugectl.c | 29 +++--
> 1 files changed, 27 insertions(+), 2 deletions(-)
>
> diff --git a/hugectl.c b/hugectl.c
> index 8ba794c..ffbd260 100644
> --- a/hugectl.c
> +++ b/hugectl.c
> @@ -36,6 +36,15 @@
> #include
> #include
>
> +#define REPORT(level, prefix, format, ...) \
> + do { \
> + if (verbose_level >= level) \
> + fprintf(stderr, "hugectl: " prefix ": " format, \
> + ##__VA_ARGS__); \
> + } while (0);
> +
> +#include "libhugetlbfs_debug.h"
> +
> extern int errno;
> extern int optind;
> extern char *optarg;
> @@ -51,6 +60,20 @@ void print_usage()
> OPTION("--help, -h", "Prints this message");
> }
>
> +int verbose_level = VERBOSITY_DEFAULT;
> +
> +void verbose_init(void)
> +{
> + char *env;
> +
> + env = getenv("HUGETLB_VERBOSE");
> + if (env)
> + verbose_level = atoi(env);
> + env = getenv("HUGETLB_DEBUG");
> + if (env)
> + verbose_level = VERBOSITY_MAX;
> +}
> +
> int main(int argc, char** argv)
> {
> char opts[] = "+h";
> @@ -60,6 +83,8 @@ int main(int argc, char** argv)
> {0},
> };
>
> + verbose_init();
> +
> while (ret != -1) {
> ret = getopt_long(argc, argv, opts, long_opts, &index);
> switch (ret) {
> @@ -72,7 +97,7 @@ int main(int argc, char** argv)
> exit(EXIT_SUCCESS);
>
> default:
> - fprintf(stderr, "unparsed option %08x\n", ret);
> + WARNING("unparsed option %08x\n", ret);
> ret = -1;
> break;
> }
> @@ -85,6 +110,6 @@ int main(int argc, char** argv)
> }
>
> execvp(argv[index], &argv[index]);
> - fprintf(stderr, "Error calling execvp: '%s'\n", strerror(errno));
> + ERROR("exec failed: %s\n", strerror(errno));
> exit(EXIT_FAILURE);
> }
--
Adam Litke - (agl at us.ibm.com)
IBM Linux Technology Center
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 5/6] hugectl: add support for requesting which library to use
On Thu, Jul 24, 2008 at 01:49:24PM -0500, Adam Litke wrote: > On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote: > > Add support for requesting a specific library set for preload. This adds > > the --library option which takes an optional argument. Without an argument > > it requests use of the specific libraries installed with the version > > of hugectl in use. An argument is treated as a prefix to the library > > directories and the approprate 32/64 bit library directories are added. > > Hmm, there is a use case that this patch doesn't cover (and it was the > first thing I thought of when I saw the patch title). Say I have a > pre-built library in /home/aglitke (or some such path without lib/lib64 > structure) that I want to use. The patch (as it exists now) would not > let me point hugectl at this library. Am I the only one who thinks this > might be useful? I suppose one option is to make users wanting to do > this simply use LD_LIBRARY_PATH directly. I can probabally sort out a way to specify that too. Will consider it further in the morning. Checking directly for the library in the passed directory name then use it directly would do the trick. -apw - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH] Allow shmget() to be overridden to add the SHM_HUGETLB flag V4
(Note, how GLIBC_2.0 and GLIB_2.3 have to be overidden is the big change in this version. It was necessary to pass all tests on PPC64) There are applications that are not hugepage-aware but use shared memory such as the postgres database. It is possible that the SHM_HUGETLB flag could be added to such an application via libhugetlbfs and overriding shmget() to add the SHM_HUGETLB to the flags. This patch enables libhugetlbfs to add the SHM_HUGETLB when a HUGETLB_SHM environment variable is set to "yes". It will work whether the application is linked to libhugetlbfs or loaded via LD_PRELOAD. To override shmget(), version.lds defines shmget for GLIBC_2.0 and GLIBC_2.3. Different versions of glibc may define this symbol with a different name requiring further updates but the test-case will fail when this situation occurs. Ideally, it would be possible to have shmget unversioned but there does not appear to be a way of specifying a specific symbol to be versioned. The local: * directive could be removed in version.lds but then all unspecified symbols would be visible externally to the library. Changelog since V3 o Do not hardcode the path to libc (Thanks to Adam Litke) o Take reserved pages into account when resizing pool during testing o Add GLIBC_2.3 to version.lds for shmget overriding on 64 bit Changelog since V2 o Link libdl to libhugetlbfs Changelog since V1 o Move shmget() from a separate library into libhugetlbfs.so o Separate out linked and unlinked testcases Signed-off-by: Mel Gorman <[EMAIL PROTECTED]> diff --git a/HOWTO b/HOWTO index b2040ed..b6a581c 100644 --- a/HOWTO +++ b/HOWTO @@ -275,6 +275,33 @@ By default, the hugepage heap does not shrink. To enable hugepage heap shrinking, set HUGETLB_MORECORE_SHRINK=yes. NB: We have been seeing some unexpected behavior from glibc's malloc when this is enabled. +Using hugepage shared memory + + +This feature allows an existing (dynamically linked) binary executable to +use hugepages for its shmget() calls. A hugepage-aware application may use +hugepages for shared memory segments by using the SHM_HUGETLB flag with +shmget(). For applications that use shmget() but do not set SHM_HUGETLB, +libhugetlbfs when preloaded can add the flag. The environment variables setup +are similar to morecore except for step 3. + +1. See step for morecore + +2. See step for morecore + +3. Set HUGETLB_SHM=yes + The shmget() call is overridden whether the application is linked or the + libhugetlbfs library is preloaded. When this environment variable is set, + the SHM_HUGETLB flag is added to the call and the size parameter is aligned + to back the shared memory segment with huge pages. In the event hugepages + cannot be used, small pages will be used instead and a warning will be + printed to explain the failure. + +The overriding assumes the version of shmget in libc is GLIBC_2.0. If +this is not the case, the overriding will not work. The situation can be +detected by using LD_DEBUG=all to see what shmget() resolves to or checking +if "make func" completes the shmget tests correctly. + Using hugepage text, data, or BSS - diff --git a/Makefile b/Makefile index d69a365..97c2c25 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ PREFIX = /usr/local EXEDIR = /bin -LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o +LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o shm.o OBJS = hugectl.o INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a INSTALL_OBJ = hugectl @@ -16,7 +16,7 @@ NODEPTARGETS= INSTALL = install -LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds +LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds -ldl CFLAGS ?= -O2 -g CFLAGS += -Wall -fPIC CPPFLAGS += -D__LIBHUGETLBFS__ diff --git a/shm.c b/shm.c new file mode 100755 index 000..8910f54 --- /dev/null +++ b/shm.c @@ -0,0 +1,92 @@ +/* + * libhugetlbfs - Easy use of Linux hugepages + * Copyright (C) 2005-2006 David Gibson & Adam Litke, IBM Corporation. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation; either version 2.1 of + * the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "libhugetlbfs_inte
[Libhugetlbfs-devel] [PATCH 0/2] Allow shmget to be overridden to add SHM_HUGETLB V5
The following two patches enable shmget() to be overridden to add the SHM_HUGETLB flag. The first patch scopes local functions specifically so that shmget() can be unversioned. The second patch adds the version of shmget() to override glibc with. Changelog since V4 o Do not use versioned shmget. As a result, this should work with any version of glibc where as previous versions of this patch were tied to GLIBC_2_0 or in a separate library Changelog since V3 o Do not hardcode the path to libc (Thanks to Adam Litke) o Take reserved pages into account when resizing pool o Add GLIBC_2.3 to version.lds for shmget overriding Changelog since V2 o Link libdl to libhugetlbfs Changelog since V1 o Move shmget() from a separate library into libhugetlbfs.so o Separate out linked and unlinked testcases -- Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [PATCH 1/2] Be specific about what local symbols should not be exported
To override shmget(), it is necessary for the symbol to be
unversioned. However, all unversioned symbols are given local scope to avoid
internal functions being called accidently. This patch marks the internal-only
export functions clearly with the prefix __lh_ and then versions them to
be only of local scope. Two unused functions are simply deleted.
Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
---
debug.c |2 +-
elflink.c |4 ++--
hugetlbfs.h |4 +---
hugeutils.c | 12 +---
init.c |6 +++---
libhugetlbfs_internal.h |6 +++---
morecore.c |6 +++---
version.lds |2 +-
8 files changed, 15 insertions(+), 27 deletions(-)
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-clean/debug.c
libhugetlbfs-gitlatest-0010-private-export/debug.c
--- libhugetlbfs-gitlatest-clean/debug.c2008-07-24 18:03:54.0
+0100
+++ libhugetlbfs-gitlatest-0010-private-export/debug.c 2008-07-24
23:03:33.0 +0100
@@ -58,7 +58,7 @@ static void __hugetlbfs_init_debug(void)
initialized = 1;
}
-void __hugetlbfs_setup_debug(void)
+void __lh_hugetlbfs_setup_debug(void)
{
__hugetlbfs_init_debug();
}
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-clean/elflink.c
libhugetlbfs-gitlatest-0010-private-export/elflink.c
--- libhugetlbfs-gitlatest-clean/elflink.c 2008-07-24 18:03:54.0
+0100
+++ libhugetlbfs-gitlatest-0010-private-export/elflink.c2008-07-24
23:03:33.0 +0100
@@ -581,7 +581,7 @@ static unsigned long hugetlb_prev_slice_
/*
* Store a copy of the given program header
*/
-int save_phdr(int table_idx, int phnum, const ElfW(Phdr) *phdr)
+static int save_phdr(int table_idx, int phnum, const ElfW(Phdr) *phdr)
{
int prot = 0;
@@ -1176,7 +1176,7 @@ static int parse_elf()
return 0;
}
-void __hugetlbfs_setup_elflink(void)
+void __lh_hugetlbfs_setup_elflink(void)
{
int i, ret;
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-clean/hugetlbfs.h
libhugetlbfs-gitlatest-0010-private-export/hugetlbfs.h
--- libhugetlbfs-gitlatest-clean/hugetlbfs.h2008-07-24 18:03:54.0
+0100
+++ libhugetlbfs-gitlatest-0010-private-export/hugetlbfs.h 2008-07-24
23:03:33.0 +0100
@@ -27,9 +27,7 @@ const char *hugetlbfs_find_path(void);
int hugetlbfs_unlinked_fd(void);
/* Diagnoses/debugging only functions */
-long hugetlbfs_num_free_pages(void);
-long hugetlbfs_num_pages(void);
-long dump_proc_pid_maps(void);
+long __lh_dump_proc_pid_maps(void);
#define PF_LINUX_HUGETLB 0x10
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-clean/hugeutils.c
libhugetlbfs-gitlatest-0010-private-export/hugeutils.c
--- libhugetlbfs-gitlatest-clean/hugeutils.c2008-07-24 18:03:54.0
+0100
+++ libhugetlbfs-gitlatest-0010-private-export/hugeutils.c 2008-07-24
23:03:33.0 +0100
@@ -264,18 +264,8 @@ int hugetlbfs_unlinked_fd(void)
/* Library user visible DIAGNOSES/DEBUGGING ONLY functions */
//
-long hugetlbfs_num_free_pages(void)
-{
- return read_meminfo("HugePages_Free:");
-}
-
-long hugetlbfs_num_pages(void)
-{
- return read_meminfo("HugePages_Total:");
-}
-
#define MAPS_BUF_SZ 4096
-long dump_proc_pid_maps()
+long __lh_dump_proc_pid_maps()
{
FILE *f;
char line[MAPS_BUF_SZ];
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-clean/init.c
libhugetlbfs-gitlatest-0010-private-export/init.c
--- libhugetlbfs-gitlatest-clean/init.c 2008-07-24 18:03:54.0 +0100
+++ libhugetlbfs-gitlatest-0010-private-export/init.c 2008-07-24
23:03:33.0 +0100
@@ -21,9 +21,9 @@
static void __attribute__ ((constructor)) setup_libhugetlbfs(void)
{
- __hugetlbfs_setup_debug();
+ __lh_hugetlbfs_setup_debug();
#ifndef NO_ELFLINK
- __hugetlbfs_setup_elflink();
+ __lh_hugetlbfs_setup_elflink();
#endif
- __hugetlbfs_setup_morecore();
+ __lh_hugetlbfs_setup_morecore();
}
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-clean/libhugetlbfs_internal.h
libhugetlbfs-gitlatest-0010-private-export/libhugetlbfs_internal.h
--- libhugetlbfs-gitlatest-clean/libhugetlbfs_internal.h2008-07-24
18:03:54.0 +0100
+++ libhugetlbfs-gitlatest-0010-private-export/libhugetlbfs_internal.h
2008-07-24 23:03:33.0 +0100
@@ -43,9 +43,9 @@
extern int __hugetlbfs_verbose;
extern int __hugetlbfs_debug;
extern int __hugetlbfs_prefault;
-extern void __hugetlbfs_setup_elflink();
-extern void __hugetlbfs_setup_morecore();
-extern void __hugetlbfs_setup_debug();
+extern void __lh_hugetlbfs_setup_elflink();
+extern void __lh_hugetlbfs_setup_morecore();
+extern void __lh_hugetlbfs_setup_debug();
extern char __
[Libhugetlbfs-devel] [PATCH 2/2] Allow shmget() to be overridden to add the SHM_HUGETLB flag
There are applications that are not hugepage-aware but use shared memory such
as the postgres database. It is possible that the SHM_HUGETLB flag could
be added to such an application via libhugetlbfs and overriding shmget()
to add the SHM_HUGETLB to the flags. This patch enables libhugetlbfs to add
the SHM_HUGETLB when a HUGETLB_SHM environment variable is set to "yes".
It will work whether the application is linked to libhugetlbfs or loaded
via LD_PRELOAD.
Signed-off-by: Mel Gorman <[EMAIL PROTECTED]>
---
HOWTO| 22 +
Makefile |4
shm.c| 81 ++
tests/Makefile |7 +
tests/run_tests.sh |4
tests/shmoverride_unlinked.c | 166 ++
6 files changed, 280 insertions(+), 4 deletions(-)
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-0010-private-export/HOWTO
libhugetlbfs-gitlatest-0020-shmget-override/HOWTO
--- libhugetlbfs-gitlatest-0010-private-export/HOWTO2008-07-24
18:03:54.0 +0100
+++ libhugetlbfs-gitlatest-0020-shmget-override/HOWTO 2008-07-24
23:03:50.0 +0100
@@ -275,6 +275,28 @@ By default, the hugepage heap does not s
shrinking, set HUGETLB_MORECORE_SHRINK=yes. NB: We have been seeing some
unexpected behavior from glibc's malloc when this is enabled.
+Using hugepage shared memory
+
+
+This feature allows an existing (dynamically linked) binary executable to
+use hugepages for its shmget() calls. A hugepage-aware application may use
+hugepages for shared memory segments by using the SHM_HUGETLB flag with
+shmget(). For applications that use shmget() but do not set SHM_HUGETLB,
+libhugetlbfs when preloaded can add the flag. The environment variables setup
+are similar to morecore except for step 3.
+
+1. See step for morecore
+
+2. See step for morecore
+
+3. Set HUGETLB_SHM=yes
+ The shmget() call is overridden whether the application is linked or the
+ libhugetlbfs library is preloaded. When this environment variable is set,
+ the SHM_HUGETLB flag is added to the call and the size parameter is aligned
+ to back the shared memory segment with huge pages. In the event hugepages
+ cannot be used, small pages will be used instead and a warning will be
+ printed to explain the failure.
+
Using hugepage text, data, or BSS
-
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-0010-private-export/Makefile
libhugetlbfs-gitlatest-0020-shmget-override/Makefile
--- libhugetlbfs-gitlatest-0010-private-export/Makefile 2008-07-24
18:03:54.0 +0100
+++ libhugetlbfs-gitlatest-0020-shmget-override/Makefile2008-07-24
23:03:33.0 +0100
@@ -1,7 +1,7 @@
PREFIX = /usr/local
EXEDIR = /bin
-LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o
+LIBOBJS = hugeutils.o version.o init.o morecore.o debug.o alloc.o shm.o
OBJS = hugectl.o
INSTALL_OBJ_LIBS = libhugetlbfs.so libhugetlbfs.a
INSTALL_OBJ = hugectl
@@ -16,7 +16,7 @@ NODEPTARGETS=
INSTALL = install
-LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds
+LDFLAGS += --no-undefined-version -Wl,--version-script=version.lds -ldl
CFLAGS ?= -O2 -g
CFLAGS += -Wall -fPIC
CPPFLAGS += -D__LIBHUGETLBFS__
diff -rup -X /usr/src/patchset-0.6/bin//dontdiff
libhugetlbfs-gitlatest-0010-private-export/shm.c
libhugetlbfs-gitlatest-0020-shmget-override/shm.c
--- libhugetlbfs-gitlatest-0010-private-export/shm.c2008-07-24
23:04:16.0 +0100
+++ libhugetlbfs-gitlatest-0020-shmget-override/shm.c 2008-07-24
23:10:03.0 +0100
@@ -0,0 +1,81 @@
+/*
+ * libhugetlbfs - Easy use of Linux hugepages
+ * Copyright (C) 2005-2006 David Gibson & Adam Litke, IBM Corporation.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public License
+ * as published by the Free Software Foundation; either version 2.1 of
+ * the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#define _GNU_SOURCE
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include "libhugetlbfs_internal.h"
+#include "hugetlbfs.h"
+
+int shmget(key_t key, size_t size, int shmflg)
+{
+ static int (*real_shmget)(key_t key, size_t size, int shmflg) = NULL;
+ char *error;
+ int retval;
+ char *hugetlbshm_env;
+ size_t alig
Re: [Libhugetlbfs-devel] [PATCH] tools: hugeedit: Modify binaries to set default remapping behavior V5
On 24.07.2008 [19:41:38 +0100], Andy Whitcroft wrote:
> On Thu, Jul 24, 2008 at 06:12:02PM +, Adam Litke wrote:
> > The recently added relinking method makes it possible to choose from
> > multiple
> > ways to remap segments at run-time via the HUGTLB_ELFMAP environment
> > variable.
> > If nothing is specified in that variable, then no remapping will occur.
> > Sometimes it is desirable to set a default remapping mode so that
> > HUGETLB_ELFMAP does not need to be set. This semantic applied to the
> > original
> > remapping algorithm.
> >
> > This patch adds a utility that can modify the PF_LINUX_HUGETLB segment
> > flags in
> > the ELF haeder of a binary to control which segments will be remapped by
> > default. The tool will also display this information.
> Looks good to me other than the silly argc thing. Tested with the patch
> below and it works a treat as far as I can tell.
>
> Tested-by: Andy Whitcroft <[EMAIL PROTECTED]>
>
> With the below merged in:
>
> Acked-by: Andy Whitcroft <[EMAIL PROTECTED]>
>
> -apw
>
> diff --git a/hugeedit.c b/hugeedit.c
> index eea0e51..fc77918 100644
> --- a/hugeedit.c
> +++ b/hugeedit.c
> @@ -199,7 +199,7 @@ int main(int argc, char ** argv)
> exit(EXIT_FAILURE);
> }
>
> - if ((argc - index) < 1) {
> + if ((argc - index) != 1) {
> print_usage();
> exit(EXIT_FAILURE);
> }
Applied with this modification.
Thanks,
Nish
--
Nishanth Aravamudan <[EMAIL PROTECTED]>
IBM Linux Technology Center
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
Libhugetlbfs-devel mailing list
Libhugetlbfs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] [ANNOUNCE] 2.0-pre1 released
Hi all, I am happy to announce the 2.0-pre1 release of libhugetlbfs. We've had a flurry of development recently and I'm trying to balance regular releases (which means regular distribution releases as well) with new features. In that vein, I've decided to postpone the following patchsets until 2.1: - Multiple mountpoints patchset (Adam Litke) - hugectl functional additions (Andy Whitcroft) - shm interface (Mel Gorman) That should allow a rapid release of 2.1 immediately following 2.0. In the interest of full disclosure, I have already applied 6 patches since 2.0-pre1, including MAP_NORESERVE for read-only mappings, a few more hugectl fixes (but should not be functional changes) and hugeedit (which is needed to enabled a default remapping mode, a current regression relative to the old relinking method). I don't plan on there being any further major commits before 2.0 can release, so I expect to tag 2.0-pre2 later this week, just to have it be available as another snapshot. The highlights for 2.0: - new relinking method (Adam Litke) - test updates Please test, test, test and test some more. As per usual, we want to see completely clean `make check` runs on x86, x86_64, power, ia64 and sparc64. If you hit any issues, please let us know on the mailing list. The diffstat/commitlog: Adam Litke (14): elflink: Handle unaligned segment starting addresses elflink: Support full segment remapping with the system linker scripts elflink: Consolidate parsing code ld.hugetlbfs: Add support for relinking with system scripts doc: Update HOWTO with new relinking instructions tests: map_high_truncate_2 needs MAP_NORESERVE tests: Add test cases for new relinking method elflink: Fix region-empty check for unaligned segment vaddrs elflink: tests: Make some library messages warnings and remove spurious '\n's tests: Skip linkhuge tests when they are broken tests: testutils copy of read_meminfo buffer is too small tests: Handle expected differences brought on by private reservations tests: Add quota test modes for untouched mmaps tests: Silence all library messages by default Andrew Hastings (1): add comment to skip_test() Andy Whitcroft (5): hugectl: reindent switch indent and fix spacing hugectl: cleanup exit handling hugectl: fix up minimum argument checks hugectl: cleanup option definitions debug: cleanup and simplify specification of debugging levels David Gibson (1): libbugetlbfs: Test case for powerpc huge_ptep_set_wrprotect() bug Eric B Munson (4): Add hugectl source Add hugectl to Makefile Add test for mounts file larger than 4kb Update large_mounts test to be compatible with multiple mount point /proc/mounts parsing Mel Gorman (3): Provide a direct allocator API for huge pages Add a regression test for get_huge_pages() Use MAP_NORESERVE for read-only mappings when HUGETLB_SHARE is enabled HOWTO | 118 +++ Makefile| 26 +- alloc.c | 132 + elflink.c | 224 +-- hugectl.c | 82 hugetlbfs.h | 13 +++ ld.hugetlbfs| 15 +++ libhugetlbfs_debug.h| 30 ++ libhugetlbfs_internal.h | 30 ++ tests/Makefile | 21 - tests/counters.c| 28 -- tests/fork-cow.c| 159 ++ tests/get_huge_pages.c | 56 +++ tests/hugetests.h |3 + tests/large_mounts.c| 104 tests/linkhuge_rw.c | 210 tests/map_high_truncate_2.c |2 +- tests/quota.c | 69 ++ tests/run_tests.sh | 103 + tests/testutils.c | 65 - version.lds |6 + 21 files changed, 1327 insertions(+), 169 deletions(-) create mode 100644 alloc.c create mode 100644 hugectl.c create mode 100644 libhugetlbfs_debug.h create mode 100644 tests/fork-cow.c create mode 100644 tests/get_huge_pages.c create mode 100644 tests/large_mounts.c create mode 100644 tests/linkhuge_rw.c -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net htt
Re: [Libhugetlbfs-devel] [PATCH 2/2] Allow shmget() to be overridden to add the SHM_HUGETLB flag
On 24.07.2008 [23:36:29 +0100], Mel Gorman wrote: > > There are applications that are not hugepage-aware but use shared memory such > as the postgres database. It is possible that the SHM_HUGETLB flag could > be added to such an application via libhugetlbfs and overriding shmget() > to add the SHM_HUGETLB to the flags. This patch enables libhugetlbfs to add > the SHM_HUGETLB when a HUGETLB_SHM environment variable is set to "yes". > It will work whether the application is linked to libhugetlbfs or loaded > via LD_PRELOAD. > > Signed-off-by: Mel Gorman <[EMAIL PROTECTED]> > --- > > HOWTO| 22 + > Makefile |4 > shm.c| 81 ++ > tests/Makefile |7 + > tests/run_tests.sh |4 > tests/shmoverride_unlinked.c | 166 ++ > 6 files changed, 280 insertions(+), 4 deletions(-) > > diff -rup -X /usr/src/patchset-0.6/bin//dontdiff > libhugetlbfs-gitlatest-0010-private-export/HOWTO > libhugetlbfs-gitlatest-0020-shmget-override/HOWTO > --- libhugetlbfs-gitlatest-0010-private-export/HOWTO 2008-07-24 > 18:03:54.0 +0100 > +++ libhugetlbfs-gitlatest-0020-shmget-override/HOWTO 2008-07-24 > 23:03:50.0 +0100 > @@ -275,6 +275,28 @@ By default, the hugepage heap does not s > shrinking, set HUGETLB_MORECORE_SHRINK=yes. NB: We have been seeing some > unexpected behavior from glibc's malloc when this is enabled. > > +Using hugepage shared memory > + > + > +This feature allows an existing (dynamically linked) binary executable to > +use hugepages for its shmget() calls. A hugepage-aware application may use > +hugepages for shared memory segments by using the SHM_HUGETLB flag with > +shmget(). For applications that use shmget() but do not set SHM_HUGETLB, > +libhugetlbfs when preloaded can add the flag. The environment variables setup > +are similar to morecore except for step 3. > + > +1. See step for morecore > + > +2. See step for morecore There may be precedent for this in the HOWTO, but I slightly prefer spelling out these steps. I don't expect them to be too long and it might be easier for users to just see everything in one place for this feature? The obvious risk is divergence, but it also means I can nag people who don't update both (including myself) when they update one. Thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 1/2] Be specific about what local symbols should not be exported
On 24.07.2008 [23:36:09 +0100], Mel Gorman wrote: > > To override shmget(), it is necessary for the symbol to be > unversioned. However, all unversioned symbols are given local scope to > avoid internal functions being called accidently. This patch marks the > internal-only export functions clearly with the prefix __lh_ and then > versions them to be only of local scope. Two unused functions are > simply deleted. I definitely prefer this to opening up all the functions. I think the two removed functions were simply there to help with tests that may want to call them, but are obviously racy. I don't know if they really do any harm being there, though? Thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 1/2] Be specific about what local symbols should not be exported
On (24/07/08 17:13), Nishanth Aravamudan didst pronounce: > On 24.07.2008 [23:36:09 +0100], Mel Gorman wrote: > > > > To override shmget(), it is necessary for the symbol to be > > unversioned. However, all unversioned symbols are given local scope to > > avoid internal functions being called accidently. This patch marks the > > internal-only export functions clearly with the prefix __lh_ and then > > versions them to be only of local scope. Two unused functions are > > simply deleted. > > I definitely prefer this to opening up all the functions. I think the > two removed functions were simply there to help with tests that may want > to call them, but are obviously racy. I don't know if they really do any > harm being there, though? > I guess they are doing no harm and I could convert them to __lh_ as well but it's dead code of trivial one-liners. They don't have a lot of point :/ -- Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 1/2] Be specific about what local symbols should not be exported
On 25.07.2008 [01:16:30 +0100], Mel Gorman wrote: > On (24/07/08 17:13), Nishanth Aravamudan didst pronounce: > > On 24.07.2008 [23:36:09 +0100], Mel Gorman wrote: > > > > > > To override shmget(), it is necessary for the symbol to be > > > unversioned. However, all unversioned symbols are given local scope to > > > avoid internal functions being called accidently. This patch marks the > > > internal-only export functions clearly with the prefix __lh_ and then > > > versions them to be only of local scope. Two unused functions are > > > simply deleted. > > > > I definitely prefer this to opening up all the functions. I think the > > two removed functions were simply there to help with tests that may want > > to call them, but are obviously racy. I don't know if they really do any > > harm being there, though? > > > > I guess they are doing no harm and I could convert them to __lh_ as well > but it's dead code of trivial one-liners. They don't have a lot of point > :/ Yeah, and given we have them in previous releases, they can be brought back easily enough. Kill 'em. -Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 5/6] hugectl: add support for requesting which library to use
On 24.07.2008 [21:48:19 +0100], Andy Whitcroft wrote: > On Thu, Jul 24, 2008 at 01:49:24PM -0500, Adam Litke wrote: > > On Thu, 2008-07-24 at 17:44 +0100, Andy Whitcroft wrote: > > > Add support for requesting a specific library set for preload. This adds > > > the --library option which takes an optional argument. Without an > > > argument > > > it requests use of the specific libraries installed with the version > > > of hugectl in use. An argument is treated as a prefix to the library > > > directories and the approprate 32/64 bit library directories are added. > > > > Hmm, there is a use case that this patch doesn't cover (and it was the > > first thing I thought of when I saw the patch title). Say I have a > > pre-built library in /home/aglitke (or some such path without lib/lib64 > > structure) that I want to use. The patch (as it exists now) would not > > let me point hugectl at this library. Am I the only one who thinks this > > might be useful? I suppose one option is to make users wanting to do > > this simply use LD_LIBRARY_PATH directly. > > I can probabally sort out a way to specify that too. Will consider it > further in the morning. > > Checking directly for the library in the passed directory name then use > it directly would do the trick. Now, I may have just missed things in the previous editions of this patch and the discussions thereafter, but I feel like this particular patch may be more complicated than it needs to be. For hugectl, I envision one of two use cases: a) Use the system's library resolution (which is either going to have been configured for ld.so via /etc/ld.so.conf or whatever (something distro-specific) or set via LD_LIBRARY_PATH by the administrator/user) to find the library, which should be the default. b) Allow the user to specify the path to the library they would like to use. That is, I dont't think we need any extra options beyond hugectl --library-path= Is there a reason for anything else? I can't think of one from my experience with users/customers. I don't think hugectl should be aware of what library it was installed with. It should just use the normal resolution methods (that is, the default library to load is not hard-coded). That should simplify the patch, as well. FWIW, the comment indicates that --library is the only option being added, but I believe there were two added in the patch. Thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
Re: [Libhugetlbfs-devel] [PATCH 4/6] hugectl: add support for preload
On 24.07.2008 [17:44:56 +0100], Andy Whitcroft wrote: > Preload is only useful when linking against an non-relinked binary > and only then when we are only attempting to back the heap. No other > combination can usefully use preload. Using preload with a relinked > binary is safe as long as you are only requesting backing of the heap. > Therefore unconditionally enable preload iff we are requesting only --heap. > As a failsafe allow the user to disable preload. Can you explain when you think a user would need to disable preload? I would expect it would only be useful for someone using hugectl with only --heap for a relinked binary, but if they are only using --heap then it should be fine to still preload? Or do I misunderstand? Thanks, Nish -- Nishanth Aravamudan <[EMAIL PROTECTED]> IBM Linux Technology Center - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
[Libhugetlbfs-devel] 代理注册外商投资企业、外商驻华商务机构、注册香港及海外公司、离岸公司、注册国内公司、注册外地驻穗办事处。
1、代理注册外商投资企业、外商驻华商务机构、注册香港及海外公司、离岸公司、注册国内公司、 注册外地驻穗办事处、注册国际及香港书刊号； 2、香港公司周年申报、年度报税，公司股份转股、公司转手、增加或减少股东服务； 3、代办香港律师公证书（用于外商投资企业之用）； 4、代理广东省内及广州企业会计报税、记帐服务； 5、代客撰写各类投标书、商业计划书、项目建议书及可行性分析报告、标准文书、法律诉讼书。 公司：广州工商注册代理公司 地址：广州市环市东路中央商务区 电话:15918644701 E-mail:[EMAIL PROTECTED] 如此邮件对您造成不便，敬请原谅。 ―― 【注意】上面的邮件内容与以下文字无关。本软件仅限于合法用途! 该邮件由《Volleymail邮件群发专家》软件发送；被网友评为最厉害 的邮件群发软件而多次要求破解！现免费下载，无限时间使用。 详情请访问我们的主页：http://www.cnysoft.com/- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/___ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
