Re: vdso feature requests from the Go people
Andy Lutomirski schreef op 13/06/2014 17:34: On Thu, Jun 12, 2014 at 10:39 PM, H. Peter Anvin wrote: On 06/12/2014 09:36 PM, Andy Lutomirski wrote: 1. Parsing the vDSO is a PITA. What if we bundled the reference parser inside the vdso? Concretely, we could have AT_VDSO_FINDENTRY point to a function like: void *vdso_find_entry(const char *name, const char *version) Then things like Go and maybe even musl (and klibc?) could just call that function. And we'd never have to worry about maintaining compatibility with more and more weird vdso parsers. Implementing this could be as simple as shoving parse_vdso.c into the vdso, although vdso2c could help and allow a really simple in-vdso implementation. I'm not really sure how much of a win that is... you have to parse *something*, and for the vast majority of all implementations there will be a dynamic linker just sitting there, and that is what it *does*. I'm only aware of two implementations that work like that: glibc and musl. AFAIK neither one even tries to use the vdso when statically linked. IIRC, Bionic doesn't support the vdso at all, and Go has the present issue. Glibc DOES use the VDSO even when it's statically linked. It uses the AT_SYSINFO_EHDR it gets from the kernel to find the VDSO and it will find any syms it needs dynamically. --Stijn -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
Andy Lutomirski schreef op 13/06/2014 7:23:
On Thu, Jun 12, 2014 at 10:15 PM, H. Peter Anvin wrote:
On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
If we were to implement both, maybe we'd actually want to provide
something like:
struct vdso_entry {
unsigned long vdso_entry_struct_size; /* so we can add fields later on */
void *func;
unsigned int max_stack; /* zero if not known */
};
How different is this really from the .dynsym table -- other than that
you have to find the latter via the dynamic segment?
The hash part is just to speed up lookup...
As far as I know, there's no reliable way to just read the dynsym
table -- the thing doesn't have a specified length, which is what
broke Go in the first place.
Parsing the ELF dynamic tables is kind of annoyingly complicated, and
understanding the format is a real PITA -- the documentation I've been
able to find is outright terrible.
--Andy
Why does it need a specified length? It's pretty easy to find, the
entries have a fixed length and the last entry in the table is NULLed
out (i.e. the d_tag is DT_NULL).
Just look for the PT_DYNAMIC segment in the program headers and iterate
over the fixed length Elf_Dyn entries to find what you're looking for.
As for ELF documentation, I've never had any problems understanding the
official specification:
http://refspecs.linuxbase.org/elf/elf.pdf
--Stijn
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
On 06/13/2014 08:34 AM, Andy Lutomirski wrote: > > I'm only aware of two implementations that work like that: glibc and > musl. AFAIK neither one even tries to use the vdso when statically > linked. IIRC, Bionic doesn't support the vdso at all, and Go has the > present issue. > I would expect uClibc to behave similarly. Bionic does, indeed, not support the vdso, but that is not for the lack of a linker but is really a shortcoming in Bionic. > And ELF parsing is a giant mess. Currently the vdso doesn't use > DT_GNU_HASH (easy to fix) but no one can safely rely on DT_GNU_HASH > being there, and DT_GNU_HASH isn't actually easier to parse. Right... and the vdso is small enough that the performance doesn't matter. However, we probably *ought* to publish DT_GNU_HASH data. >>> 2. Go uses a segmented stack, and the vdso is quite unfriendly for >>> segmented stack. If we can get compiler support, is there a >>> reasonable way that we could advertise the maximum stack usage of each >>> vdso entry point? >> >> I suspect an easier way to do that would just be to define a maximum >> stack usage for *any* vdso entry point, and then enable the gcc stack >> depth warning (perhaps even with Werror)... we can do this now. > > I can imagine this causing lots of pain when gcc 4.11 comes out with > some issue that blows up the stack usage. Or when akpm compiles on > Fedora Core 6 using some ancient toolchain that spills every local > variable three or four times and assigns every possible inline > function its own non-overlapping stack range. > > My copy of gcc supports -fstack-usage, which seems like an easyish way > to obtain the information. I'm not entirely sure whether > -fstack-usage refers to the whole call tree or just to the specific > function. There are issues either way. However, most vdso code doesn't use much stack at all, and it seems reasonable to put a (conservative) cap on it as a matter of policy. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
On Thu, Jun 12, 2014 at 10:39 PM, H. Peter Anvin wrote: > On 06/12/2014 09:36 PM, Andy Lutomirski wrote: >> >> 1. Parsing the vDSO is a PITA. What if we bundled the reference >> parser inside the vdso? Concretely, we could have AT_VDSO_FINDENTRY >> point to a function like: >> >> void *vdso_find_entry(const char *name, const char *version) >> >> Then things like Go and maybe even musl (and klibc?) could just call >> that function. And we'd never have to worry about maintaining >> compatibility with more and more weird vdso parsers. >> >> Implementing this could be as simple as shoving parse_vdso.c into the >> vdso, although vdso2c could help and allow a really simple in-vdso >> implementation. >> > > I'm not really sure how much of a win that is... you have to parse > *something*, and for the vast majority of all implementations there will > be a dynamic linker just sitting there, and that is what it *does*. I'm only aware of two implementations that work like that: glibc and musl. AFAIK neither one even tries to use the vdso when statically linked. IIRC, Bionic doesn't support the vdso at all, and Go has the present issue. And ELF parsing is a giant mess. Currently the vdso doesn't use DT_GNU_HASH (easy to fix) but no one can safely rely on DT_GNU_HASH being there, and DT_GNU_HASH isn't actually easier to parse. My point is that, for things that otherwise need to carry around a full ELF loader, having a really easy, guaranteed-correct way to use the vdso would be useful. I can see how much text size it would add. If the parser were part of the vdso, it could probably be trimmed down a lot. For example, there is currently exactly one version definition, and it could be hard-coded. If someone were to add another version definition, they could fix the parser at the same time. Basically, everything except for the vdso_sym function in my parser could go. > >> 2. Go uses a segmented stack, and the vdso is quite unfriendly for >> segmented stack. If we can get compiler support, is there a >> reasonable way that we could advertise the maximum stack usage of each >> vdso entry point? > > I suspect an easier way to do that would just be to define a maximum > stack usage for *any* vdso entry point, and then enable the gcc stack > depth warning (perhaps even with Werror)... we can do this now. I can imagine this causing lots of pain when gcc 4.11 comes out with some issue that blows up the stack usage. Or when akpm compiles on Fedora Core 6 using some ancient toolchain that spills every local variable three or four times and assigns every possible inline function its own non-overlapping stack range. My copy of gcc supports -fstack-usage, which seems like an easyish way to obtain the information. I'm not entirely sure whether -fstack-usage refers to the whole call tree or just to the specific function. --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
On 06/12/2014 09:36 PM, Andy Lutomirski wrote: > > 1. Parsing the vDSO is a PITA. What if we bundled the reference > parser inside the vdso? Concretely, we could have AT_VDSO_FINDENTRY > point to a function like: > > void *vdso_find_entry(const char *name, const char *version) > > Then things like Go and maybe even musl (and klibc?) could just call > that function. And we'd never have to worry about maintaining > compatibility with more and more weird vdso parsers. > > Implementing this could be as simple as shoving parse_vdso.c into the > vdso, although vdso2c could help and allow a really simple in-vdso > implementation. > I'm not really sure how much of a win that is... you have to parse *something*, and for the vast majority of all implementations there will be a dynamic linker just sitting there, and that is what it *does*. > 2. Go uses a segmented stack, and the vdso is quite unfriendly for > segmented stack. If we can get compiler support, is there a > reasonable way that we could advertise the maximum stack usage of each > vdso entry point? I suspect an easier way to do that would just be to define a maximum stack usage for *any* vdso entry point, and then enable the gcc stack depth warning (perhaps even with Werror)... we can do this now. -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
On 06/12/2014 10:23 PM, Andy Lutomirski wrote: > > As far as I know, there's no reliable way to just read the dynsym > table -- the thing doesn't have a specified length, which is what > broke Go in the first place. > Ah yes, you're right. > > Parsing the ELF dynamic tables is kind of annoyingly complicated, and > understanding the format is a real PITA -- the documentation I've been > able to find is outright terrible. > Yes... not entirely clear that putting it in the vdso is a net win, though. There are some people who would like to put a significant chunk of libc in the vdso, and it just doesn't seem like a good idea. An index table is somewhat more reasonable, but (a) not backwards compatible (so wouldn't work with older kernels) and (b) has shades of Windows... -hpa -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
On Thu, Jun 12, 2014 at 10:15 PM, H. Peter Anvin wrote:
> On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
>>
>> If we were to implement both, maybe we'd actually want to provide
>> something like:
>>
>> struct vdso_entry {
>> unsigned long vdso_entry_struct_size; /* so we can add fields later on */
>> void *func;
>> unsigned int max_stack; /* zero if not known */
>> };
>>
>
> How different is this really from the .dynsym table -- other than that
> you have to find the latter via the dynamic segment?
>
> The hash part is just to speed up lookup...
As far as I know, there's no reliable way to just read the dynsym
table -- the thing doesn't have a specified length, which is what
broke Go in the first place.
Parsing the ELF dynamic tables is kind of annoyingly complicated, and
understanding the format is a real PITA -- the documentation I've been
able to find is outright terrible.
--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
On 06/12/2014 09:36 PM, Andy Lutomirski wrote:
>
> If we were to implement both, maybe we'd actually want to provide
> something like:
>
> struct vdso_entry {
> unsigned long vdso_entry_struct_size; /* so we can add fields later on */
> void *func;
> unsigned int max_stack; /* zero if not known */
> };
>
How different is this really from the .dynsym table -- other than that
you have to find the latter via the dynamic segment?
The hash part is just to speed up lookup...
-hpa
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: vdso feature requests from the Go people
On Thu, Jun 12, 2014 at 9:36 PM, Andy Lutomirski wrote:
> I was talking to some of the Go people, and they have a couple of IMO
> reasonable feature requests.
>
> 1. Parsing the vDSO is a PITA. What if we bundled the reference
> parser inside the vdso? Concretely, we could have AT_VDSO_FINDENTRY
> point to a function like:
>
> void *vdso_find_entry(const char *name, const char *version)
>
> Then things like Go and maybe even musl (and klibc?) could just call
> that function. And we'd never have to worry about maintaining
> compatibility with more and more weird vdso parsers.
>
> Implementing this could be as simple as shoving parse_vdso.c into the
> vdso, although vdso2c could help and allow a really simple in-vdso
> implementation.
>
> 2. Go uses a segmented stack, and the vdso is quite unfriendly for
> segmented stack. If we can get compiler support, is there a
> reasonable way that we could advertise the maximum stack usage of each
> vdso entry point?
>
>
> If we were to implement both, maybe we'd actually want to provide
> something like:
>
> struct vdso_entry {
> unsigned long vdso_entry_struct_size; /* so we can add fields later on */
> void *func;
> unsigned int max_stack; /* zero if not known */
> };
>
> vdso_entry *vdso_find_entry(const char *name, const char *version);
>
> AT_VDSO_FINDENTRY would be set to &vdso_find_entry.
>
> This shouldn't even break CRIU :)
>
Alternatively we could just assign each entry a stable index and
supply a table: AT_VDSO_ENTRIES would point to:
struct {
unsigned long nentries;
vdso_entry entries[];
}
Simpler, faster, but less flexible. This version wouldn't even need
help from vdso2c.
--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
vdso feature requests from the Go people
I was talking to some of the Go people, and they have a couple of IMO
reasonable feature requests.
1. Parsing the vDSO is a PITA. What if we bundled the reference
parser inside the vdso? Concretely, we could have AT_VDSO_FINDENTRY
point to a function like:
void *vdso_find_entry(const char *name, const char *version)
Then things like Go and maybe even musl (and klibc?) could just call
that function. And we'd never have to worry about maintaining
compatibility with more and more weird vdso parsers.
Implementing this could be as simple as shoving parse_vdso.c into the
vdso, although vdso2c could help and allow a really simple in-vdso
implementation.
2. Go uses a segmented stack, and the vdso is quite unfriendly for
segmented stack. If we can get compiler support, is there a
reasonable way that we could advertise the maximum stack usage of each
vdso entry point?
If we were to implement both, maybe we'd actually want to provide
something like:
struct vdso_entry {
unsigned long vdso_entry_struct_size; /* so we can add fields later on */
void *func;
unsigned int max_stack; /* zero if not known */
};
vdso_entry *vdso_find_entry(const char *name, const char *version);
AT_VDSO_FINDENTRY would be set to &vdso_find_entry.
This shouldn't even break CRIU :)
--Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
