Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Sep 18, 2009, at 9:10 AM, David Laight wrote: It is all rather similar to the proliferation of memory pools - for things where 'malloc' would be fine. The old-school kernel malloc was almost never "fine" ... but pools have another advantage -- use of a direct-mapped segment on architectures so-equipped. And for those not so-equipped, greatly reduced pressure on kmem_map. Now, if we could only get rid of kmem_map / kernel malloc completely (which requires making it illegal to allocate memory in interrupt context). -- thorpej
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Thu, Sep 17, 2009 at 09:05:38AM +0100, Nick Hudson wrote: > > > People are most welcome to fix this ugliness > > properly by helping to get rid of link sets in the kernel. > > I'm glad you agree that link sets are ugly. On the matter of a getting rid of > them I did offer to discuss a solution, but was ignored. Like all things they have their place, but overuse can be a problem. For instance they could solve the problem of getting an intial entry point (code or data) into a kernel 'module' that has being linked into a kernel with 'ld -r'. (when modules are loaded it is possibly to do a symbol lookup...) There are probably other places where it is difficult to call an explicit initialiser. It is all rather similar to the proliferation of memory pools - for things where 'malloc' would be fine. David -- David Laight: da...@l8s.co.uk
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Wednesday 16 September 2009 04:08:16 Valeriy E. Ushakov wrote: [...] > PS: It's really amazing how rump can cope with link sets in the DSO > world at all. One can condescentingly call it a hack, but link sets > are not rump's fault. s/amazing/lucky/ IMO, rump is fragile and not only regularly breaks the build, but changes made for the benefit of rump break kernels[1]. It appears to be useful technology, but there is, again IMO, just too much avoidable fallout. If the "inventive use of" __start/_stop symbols was documented maybe I would have spotted it sooner. I admit that it took me a while to understand what was going on and, yes, I still think it is "inventive". > People are most welcome to fix this ugliness > properly by helping to get rid of link sets in the kernel. I'm glad you agree that link sets are ugly. On the matter of a getting rid of them I did offer to discuss a solution on ICB, but was ignored. Nick [1] http://mail-index.netbsd.org/source-changes-d/2009/08/11/msg000757.html
Re: removing link sets (Re: CVS commit: src/usr.sbin/puffs/rump_smbfs)
On Sep 16, 2009, at 8:43 AM, Eric Haszlakiewicz wrote: On Wed, Sep 16, 2009 at 07:08:16AM +0400, Valeriy E. Ushakov wrote: PS: It's really amazing how rump can cope with link sets in the DSO world at all. One can condescentingly call it a hack, but link sets are not rump's fault. People are most welcome to fix this ugliness properly by helping to get rid of link sets in the kernel. getting rid of link sets? I thought they were added specifically to fix the need to have a bunch of ifdefs in code to initialize bits of code. What do you replace them with? Yes, I came to regret the addition of them. They solved a problem at one time, but are not really a great solution. I'd like to see them go, replaced with real constructor methods for each subsystem. eric -- thorpej
removing link sets (Re: CVS commit: src/usr.sbin/puffs/rump_smbfs)
On Wed, Sep 16, 2009 at 07:08:16AM +0400, Valeriy E. Ushakov wrote: > PS: It's really amazing how rump can cope with link sets in the DSO > world at all. One can condescentingly call it a hack, but link sets > are not rump's fault. People are most welcome to fix this ugliness > properly by helping to get rid of link sets in the kernel. getting rid of link sets? I thought they were added specifically to fix the need to have a bunch of ifdefs in code to initialize bits of code. What do you replace them with? eric
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Fri, Sep 11, 2009 at 14:47:45 +1000, Geoff Wing wrote: > On Thursday 2009-09-10 13:55 +0300, Antti Kantee output: > :No, that is *absolutely the wrong thing*, since while it might make > :the build work, it breaks the resulting binary. I'm a bit baffled that > :breakage like that was committed to rump_nfs/Makefile in the first place. > : > :It seems there is a regression in binutils 2.19 which prevents the > :standard DOMAIN_DEFINE() macro from working. I suggest reverting back > :to 2.16 until the cause is identified and the bug is fixed. > > No, that's a red herring. The problem is a long-standing one with > sys/kern/uipc_domain.c (from rev 1.49 2005/01/23 onwards). > > Someone has it use link sets without ever creating a section header so > that they actually work properly. This caused librumpnet.so to also > have a bad copy. No, uipc_domain.c does not need to create the link set section because other files in the kernel do. In rump world things get more tricky because uipc_domain.c and files that actually define domains live in different DSOs. So uipc_domain.o has unresolved refs to start/stop symbols, which is ok. The real problem, as pooka explained further down the thread, is that the start/stop symbols were no longer generated in the component that actually defined domains because no file in that component actually referenced them (uipc_domain.o is in another DSO) and new binutils changed the way those symbols are created. PS: It's really amazing how rump can cope with link sets in the DSO world at all. One can condescentingly call it a hack, but link sets are not rump's fault. People are most welcome to fix this ugliness properly by helping to get rid of link sets in the kernel. SY, Uwe -- u...@stderr.spb.ru | Zu Grunde kommen http://snark.ptc.spbu.ru/~uwe/ | Ist zu Grunde gehen
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
> > The problem is that the linker suddenly changed behaviour and no longer
> > provided the symbols which used to satisfy those unresolved symbols. I have
> > convinced the linker to once again provide those symbols hence fixing the
> > build.
>
> I think you need to do some more testing and (re-)read the email from Geoff
> Wing which hinted at the problem. Nothing in librumpnet is creating a
> link_set_domain section and therefore the linker (2.16 or 2.19.1) doesn't
> create {__start,__stop}_link_set_domain symbols.
You failed to quote the line from the previous email where I already
stated this was not the issue. The symbols are not supposed to come
from librumpnet, but rather e.g. librumpnet_sockin and that is where
the new binutils behaviour was creating damage. Feel free to use nm on
e.g. librumpnet_sockin before and after my change from last night.
But if you refuse to understand the problem, fine. Just revert your
misguided changes.
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
[This will be my last email on this subject]
On Monday 14 September 2009 12:29:15 Antti Kantee wrote:
> On Mon Sep 14 2009 at 12:04:18 +0100, Nick Hudson wrote:
> > > rump hasn't relied on linksets since August 2008 (and even then there
> > > was only a few day window of linkset use after introducing dynamic
> > > linking to rump). The only thing that matters is the entry point,
> > > which, incidentally, is what binutils 2.19 chaged.
> >
> > I believe you've analysed this completely incorrectly. afaict, librumnet
> > has always had unresolved references to
> > {__start,__stop)_link_set_domains. The change from binutils 2.16 to 2.19
> > is, I believe, that 2.19 is less forgiving to this error.
> >
> >From NEWS in the ld distribution:
>
> === SNIP ===
> Changes in version 2.6:
>
> * When an ELF section name is representable as a C identifier (this is not
> true of most ELF section names), the linker will automatically define
> symbols __start_SECNAME and __stop_SECNAME, where SECNAME is the section
> name, at the beginning and the end of the section. This is used by glibc.
>
> Addendum: Current versions of the linker (at least for version 2.18
> onwards and possibly much earlier as well) place two restrictions on this
> feature: The symbols are only implemented for orphaned sections, not for
> explicitly placed sections and they are PROVIDEd rather than being defined.
> === SNIP ===
>
> As you can easily check from even our cvs tree, "Addendum" was not
> present in binutils 2.16.
Yes, I agree the *documentation* was changed between 2.16 and 2.19.1. The
*code*, however, has stayed approximately the same
> The problem is that the linker suddenly changed behaviour and no longer
> provided the symbols which used to satisfy those unresolved symbols. I have
> convinced the linker to once again provide those symbols hence fixing the
> build.
I think you need to do some more testing and (re-)read the email from Geoff
Wing which hinted at the problem. Nothing in librumpnet is creating a
link_set_domain section and therefore the linker (2.16 or 2.19.1) doesn't
create {__start,__stop}_link_set_domain symbols.
Nick
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Mon Sep 14 2009 at 12:04:18 +0100, Nick Hudson wrote:
> > rump hasn't relied on linksets since August 2008 (and even then there was
> > only a few day window of linkset use after introducing dynamic linking
> > to rump). The only thing that matters is the entry point, which,
> > incidentally, is what binutils 2.19 chaged.
>
> I believe you've analysed this completely incorrectly. afaict, librumnet has
> always had unresolved references to {__start,__stop)_link_set_domains. The
> change from binutils 2.16 to 2.19 is, I believe, that 2.19 is less forgiving
> to this error.
>From NEWS in the ld distribution:
=== SNIP ===
Changes in version 2.6:
* When an ELF section name is representable as a C identifier (this is not true
of most ELF section names), the linker will automatically define symbols
__start_SECNAME and __stop_SECNAME, where SECNAME is the section name, at the
beginning and the end of the section. This is used by glibc.
Addendum: Current versions of the linker (at least for version 2.18 onwards
and possibly much earlier as well) place two restrictions on this feature: The
symbols are only implemented for orphaned sections, not for explicitly placed
sections and they are PROVIDEd rather than being defined.
=== SNIP ===
As you can easily check from even our cvs tree, "Addendum" was not
present in binutils 2.16.
The issue is not with unresolved symbols in libraries which are going to
be linked (*sheesh*!). The problem is that the linker suddenly changed
behaviour and no longer provided the symbols which used to satisfy those
unresolved symbols. I have convinced the linker to once again provide
those symbols hence fixing the build.
(No, those symbols are not even used by rump. But what this build error
told me is that __start_link_set_modules was no longer generated either
and hence dlsym() for it does not work, and hence rump component dynamic
loading would fail. That is why it is important to not just sweep stuff
like this under the carpet.)
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Monday 14 September 2009 10:20:52 Antti Kantee wrote:
> On Mon Sep 14 2009 at 08:15:36 +0100, Nick Hudson wrote:
[...]
> > > The binutils regression is that between 2.16 and 2.19 __start_sectname
> > > for orphaned sections was changed to use PROVIDE instead of define and
> > > hence those symbols are not available anymore in library components
> > > produced with ld. I have provided a quick workaround which allows the
> > > build to work with both with 2.16 and 2.19 while not silently breaking
> > > functionality like in your model.
> > >
> > > Please revert all of your incorrect commits.
> >
> > Feel free to fix rump and linksets correctly. It's been explained to you
> > why they can't work properly together.
>
> rump hasn't relied on linksets since August 2008 (and even then there was
> only a few day window of linkset use after introducing dynamic linking
> to rump). The only thing that matters is the entry point, which,
> incidentally, is what binutils 2.19 chaged.
I believe you've analysed this completely incorrectly. afaict, librumnet has
always had unresolved references to {__start,__stop)_link_set_domains. The
change from binutils 2.16 to 2.19 is, I believe, that 2.19 is less forgiving
to this error.
Nick
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Mon Sep 14 2009 at 08:15:36 +0100, Nick Hudson wrote: > > In the future if you run across a problem which you are unable to > > understand, debug and fix on your own, please ask for assistance. > > Commits with handwavy and incorrect information make it more difficult > > to track down and fix the real issues. > > I tried to contact you on several occasions, but you chose to ignore me. > *shrug* I'm sorry, then. The commit messages and this are the only emails I've seen from you on the subject. Maybe you should quote something like "no comment from foo" in a commit message in the future? > > The binutils regression is that between 2.16 and 2.19 __start_sectname > > for orphaned sections was changed to use PROVIDE instead of define and > > hence those symbols are not available anymore in library components > > produced with ld. I have provided a quick workaround which allows the > > build to work with both with 2.16 and 2.19 while not silently breaking > > functionality like in your model. > > > > Please revert all of your incorrect commits. > > Feel free to fix rump and linksets correctly. It's been explained to you why > they can't work properly together. rump hasn't relied on linksets since August 2008 (and even then there was only a few day window of linkset use after introducing dynamic linking to rump). The only thing that matters is the entry point, which, incidentally, is what binutils 2.19 chaged.
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Sunday 13 September 2009 23:57:17 Antti Kantee wrote: > On Fri Sep 11 2009 at 06:48:50 +, Nick Hudson wrote: > > Module Name:src > > Committed By: skrll > > Date: Fri Sep 11 06:48:50 UTC 2009 > > > > Modified Files: > > src/usr.sbin/puffs/rump_smbfs: Makefile > > > > Log Message: > > Allow this to build while link sets and rump are broken. > > Hi Nick, > > In the future if you run across a problem which you are unable to > understand, debug and fix on your own, please ask for assistance. > Commits with handwavy and incorrect information make it more difficult > to track down and fix the real issues. I tried to contact you on several occasions, but you chose to ignore me. *shrug* > The binutils regression is that between 2.16 and 2.19 __start_sectname > for orphaned sections was changed to use PROVIDE instead of define and > hence those symbols are not available anymore in library components > produced with ld. I have provided a quick workaround which allows the > build to work with both with 2.16 and 2.19 while not silently breaking > functionality like in your model. > > Please revert all of your incorrect commits. Feel free to fix rump and linksets correctly. It's been explained to you why they can't work properly together. Thanks, Nick
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Fri Sep 11 2009 at 06:48:50 +, Nick Hudson wrote: > Module Name: src > Committed By: skrll > Date: Fri Sep 11 06:48:50 UTC 2009 > > Modified Files: > src/usr.sbin/puffs/rump_smbfs: Makefile > > Log Message: > Allow this to build while link sets and rump are broken. Hi Nick, In the future if you run across a problem which you are unable to understand, debug and fix on your own, please ask for assistance. Commits with handwavy and incorrect information make it more difficult to track down and fix the real issues. The binutils regression is that between 2.16 and 2.19 __start_sectname for orphaned sections was changed to use PROVIDE instead of define and hence those symbols are not available anymore in library components produced with ld. I have provided a quick workaround which allows the build to work with both with 2.16 and 2.19 while not silently breaking functionality like in your model. Please revert all of your incorrect commits. thanks
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Thursday 2009-09-10 13:55 +0300, Antti Kantee output:
:No, that is *absolutely the wrong thing*, since while it might make
:the build work, it breaks the resulting binary. I'm a bit baffled that
:breakage like that was committed to rump_nfs/Makefile in the first place.
:
:It seems there is a regression in binutils 2.19 which prevents the
:standard DOMAIN_DEFINE() macro from working. I suggest reverting back
:to 2.16 until the cause is identified and the bug is fixed.
No, that's a red herring. The problem is a long-standing one with
sys/kern/uipc_domain.c (from rev 1.49 2005/01/23 onwards).
Someone has it use link sets without ever creating a section header so
that they actually work properly. This caused librumpnet.so to also
have a bad copy. With the following initial patch, you should see
a link section with "objdump -h". The second patch just removes the
other wrong build stuff.
Regards,
Geoff
Index: sys/kern/uipc_domain.c
===
RCS file: /cvsroot/src/sys/kern/uipc_domain.c,v
retrieving revision 1.83
diff -u -r1.83 uipc_domain.c
--- sys/kern/uipc_domain.c 8 Sep 2009 18:01:34 - 1.83
+++ sys/kern/uipc_domain.c 11 Sep 2009 04:25:50 -
@@ -76,6 +76,9 @@
static struct sysctllog *domain_sysctllog;
static void sysctl_net_setup(void);
+static struct domain dummy_static_domain;
+__link_set_add_bss(domains, dummy_static_domain);
+
void
domaininit(bool addroute)
{
Index: usr.sbin/puffs/rump_nfs/Makefile
===
RCS file: /cvsroot/src/usr.sbin/puffs/rump_nfs/Makefile,v
retrieving revision 1.4
diff -u -r1.4 Makefile
--- usr.sbin/puffs/rump_nfs/Makefile21 Aug 2009 14:41:22 - 1.4
+++ usr.sbin/puffs/rump_nfs/Makefile11 Sep 2009 04:26:11 -
@@ -20,9 +20,6 @@
#LDADD+= -lrumpfs_nfs -lrumpnet -lrumpnet_net -lrumpnet_netinet \
-lrumpnet_virtif
-LDFLAGS+= -Wl,--ignore-unresolved-symbol,__start_link_set_domains
-LDFLAGS+= -Wl,--ignore-unresolved-symbol,__stop_link_set_domains
-
ISRUMP=# gives me nightmares
#CPPFLAGS+=-DINET6 # for getnfsargs?
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Thu Sep 10 2009 at 14:14:03 +1000, Geoff Wing wrote:
> On Sunday 2009-09-06 20:26 +, Antti Kantee output:
> :Module Name: src
> :Committed By:pooka
> :Date:Sun Sep 6 20:26:47 UTC 2009
> :Added Files:
> : src/usr.sbin/puffs/rump_smbfs: Makefile rump_smbfs.8 rump_smbfs.c
> : smb_rumpops.c
> :Log Message:
> :Add rump_smbfs.
> [...]
> :cvs rdiff -u -r0 -r1.1 src/usr.sbin/puffs/rump_smbfs/Makefile \
> [...]
>
> When I did a build, it seemed to need the following patch to link
> (stolen from rump_nfs/Makefile). Is it correct here?
No, that is *absolutely the wrong thing*, since while it might make
the build work, it breaks the resulting binary. I'm a bit baffled that
breakage like that was committed to rump_nfs/Makefile in the first place.
It seems there is a regression in binutils 2.19 which prevents the
standard DOMAIN_DEFINE() macro from working. I suggest reverting back
to 2.16 until the cause is identified and the bug is fixed.
> Regards,
> Geoff
>
> PS. ``descualified''?
>
> Index: Makefile
> ===
> RCS file: /cvsroot/src/usr.sbin/puffs/rump_smbfs/Makefile,v
> retrieving revision 1.1
> diff -u -r1.1 Makefile
> --- Makefile 6 Sep 2009 20:26:47 - 1.1
> +++ Makefile 10 Sep 2009 04:07:20 -
> @@ -9,6 +9,9 @@
> LDADD+= -lrumpnet_netinet -lrumpnet_sockin -lrumpnet
> LDADD+= -lrumpcrypto -lrumpdev_netsmb -lrumpdev
>
> +LDFLAGS+= -Wl,--ignore-unresolved-symbol,__start_link_set_domains
> +LDFLAGS+= -Wl,--ignore-unresolved-symbol,__stop_link_set_domains
> +
> ISRUMP= # it is
>
> .include "${.CURDIR}/../../../sbin/mount_smbfs/Makefile.inc"
Re: CVS commit: src/usr.sbin/puffs/rump_smbfs
On Sunday 2009-09-06 20:26 +, Antti Kantee output:
:Module Name: src
:Committed By: pooka
:Date: Sun Sep 6 20:26:47 UTC 2009
:Added Files:
: src/usr.sbin/puffs/rump_smbfs: Makefile rump_smbfs.8 rump_smbfs.c
: smb_rumpops.c
:Log Message:
:Add rump_smbfs.
[...]
:cvs rdiff -u -r0 -r1.1 src/usr.sbin/puffs/rump_smbfs/Makefile \
[...]
When I did a build, it seemed to need the following patch to link
(stolen from rump_nfs/Makefile). Is it correct here?
Regards,
Geoff
PS. ``descualified''?
Index: Makefile
===
RCS file: /cvsroot/src/usr.sbin/puffs/rump_smbfs/Makefile,v
retrieving revision 1.1
diff -u -r1.1 Makefile
--- Makefile6 Sep 2009 20:26:47 - 1.1
+++ Makefile10 Sep 2009 04:07:20 -
@@ -9,6 +9,9 @@
LDADD+=-lrumpnet_netinet -lrumpnet_sockin -lrumpnet
LDADD+=-lrumpcrypto -lrumpdev_netsmb -lrumpdev
+LDFLAGS+= -Wl,--ignore-unresolved-symbol,__start_link_set_domains
+LDFLAGS+= -Wl,--ignore-unresolved-symbol,__stop_link_set_domains
+
ISRUMP=# it is
.include "${.CURDIR}/../../../sbin/mount_smbfs/Makefile.inc"
