Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-13 Thread Eduardo Horvath
On Sat, 11 Aug 2012, Matt Thomas wrote: On Aug 11, 2012, at 10:35 AM, Thor Lancelot Simon wrote: On Sat, Aug 11, 2012 at 06:45:12AM +, Christos Zoulas wrote: It is a slippery slope, but I think in this case it is wise to bend. If we cannot reach agreement here, consult core.

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Martin Husemann
So, I see various things coming together in this thread, and I would suggest to fix them all with a change beyound what Emmanuel originaly proposed: - the pthread_self() usage and the binding to a register is a property of our libpthread implementation, so it should overwrite setcontext and

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Joerg Sonnenberger
On Sun, Aug 12, 2012 at 10:02:48AM +0200, Martin Husemann wrote: - however, we should bring back (under slightly different names, and in libc) the old pure userland setcontext (and friends) that we used to have in the N:M libpthread. We have the code for all archs, it is tested, and

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: Therefore I do not understand why you oppose the proposal of proposing a MI interface to let the user opt out of that bad behavior. Your fix doesn't change this issue at all. Of course it does. For now if you want to swapcontext()

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Joerg Sonnenberger
On Sun, Aug 12, 2012 at 03:00:50AM +0200, Emmanuel Dreyfus wrote: Joerg Sonnenberger jo...@britannica.bec.de wrote: Therefore I do not understand why you oppose the proposal of proposing a MI interface to let the user opt out of that bad behavior. Your fix doesn't change this issue

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: They are still very heavy for coroutine usage. I cannot argue on that point. This is where it would be interesting if you could defend that on gluster-de...@nongnu.org. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: No, it doesn't. You don't change the lwpid if you move to a different kernel thread. As such, you are breaking the lwpid is constant property. This is how NetBSD currently behave, unless the user takes care of unsetting _UC_TLSBASE before

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Joerg Sonnenberger
On Sun, Aug 12, 2012 at 06:19:02PM +0200, Emmanuel Dreyfus wrote: Joerg Sonnenberger jo...@britannica.bec.de wrote: No, it doesn't. You don't change the lwpid if you move to a different kernel thread. As such, you are breaking the lwpid is constant property. This is how NetBSD

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Emmanuel Dreyfus
rJoerg Sonnenberger jo...@britannica.bec.de wrote: You are mixing up two unrelated items. The thread private area and the thread ID are not the same thing. Your fix only affects the former, not the latter. Right, I was not sure what you were talking about. You mean struct lwp's l_lid, right?

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Joerg Sonnenberger
On Sun, Aug 12, 2012 at 07:27:36PM +0200, Emmanuel Dreyfus wrote: rJoerg Sonnenberger jo...@britannica.bec.de wrote: You are mixing up two unrelated items. The thread private area and the thread ID are not the same thing. Your fix only affects the former, not the latter. Right, I was

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: Or libpthread or rtld for the same kind of issue. I could understand libpthread would get confused if pthread_self() did not match struct lwp's l_lid, and this is just the problem I want to address by having _UC_TLSBASE on all ports. If you have

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread YAMAMOTO Takashi
hi, On Fri, Aug 10, 2012 at 07:31:59PM +0200, Emmanuel Dreyfus wrote: Joerg Sonnenberger jo...@britannica.bec.de wrote: I maintain that trying to move contexts between threads is an inherently bad idea and that it is a very inefficient interface for implementing coroutines. I object to

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-12 Thread Emmanuel Dreyfus
YAMAMOTO Takashi y...@mwd.biglobe.ne.jp wrote: the current bahaviour would benefit applications which use i386 %gs register for their own purposes. is it your concern? Note I did not propose to alter the default behavior. I just want an option to disable it. -- Emmanuel Dreyfus

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread David Holland
On Fri, Aug 10, 2012 at 08:56:42PM +0200, Emmanuel Dreyfus wrote: Only by addressing the complete picture can we reasonably expect to decide on a semantically sound interface that we won't regret in the future. Short reply, I will address the rest later: We already have the

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
David Holland dholland-t...@netbsd.org wrote: (1) What is the CV of the pertinent standards that describe setcontext and getcontext? As explained in their man page, they are obsolete in the 2004 edition of POSIX.1, and were removed in the 2008 edition. Standards are of little help here. (2)

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Christos Zoulas
In article 20120810173818.ga8...@britannica.bec.de, Joerg Sonnenberger jo...@britannica.bec.de wrote: On Fri, Aug 10, 2012 at 07:31:59PM +0200, Emmanuel Dreyfus wrote: Joerg Sonnenberger jo...@britannica.bec.de wrote: I maintain that trying to move contexts between threads is an inherently

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Christos Zoulas chris...@astron.com wrote: Like it or not most of the world has turned into linux. We can either provide compatibility where possible (and not overly disgusting) to gain compatibility with 3rd party code developed for linux, or simply say tough, it will not work on NetBSD

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Mouse
[...], or simply say tough, it will not work on NetBSD because we refuse to compromise. IMO here it is even a worse stance, since we already have the desired fix for amd64, i386, m68k, mips, vax, and hppa. Ah, but is it really a fix? IMO this all hinges on just what the interface contract

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Christos Zoulas
On Aug 11, 11:16am, m...@netbsd.org (Emmanuel Dreyfus) wrote: -- Subject: Re: [RFC][PATCH] _UC_TLSBASE for all ports | Christos Zoulas chris...@astron.com wrote: | | Like it or not most of the world has turned into linux. We can either | provide compatibility where possible (and not overly

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Christos Zoulas chris...@zoulas.com wrote: I could believe that, if the change suggested to make this the default behavior (which some would argue it should be...) In an ideal world, we would set _UC_TLSBASE by default (as it is today, except on powerpc), and we would automatically ignore

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Martin Husemann
On Sat, Aug 11, 2012 at 12:40:44PM +0200, Emmanuel Dreyfus wrote: In an ideal world, we would set _UC_TLSBASE by default (as it is today, except on powerpc), and we would automatically ignore _UC_TLSBASE when l-l_proc-p_nlwps 1, since we cannot think of a sane usage for that. I would agree

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Martin Husemann mar...@duskware.de wrote: However, for a dummie like me who hasn't seen the code in question, could you please explain how they handle the stack pointer in the context? - get current ucontext_t wht getcontext(); - allocate a stack with malloc(), add it to ucontext_t (uc_stack

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Martin Husemann
On Sat, Aug 11, 2012 at 01:50:58PM +0200, Emmanuel Dreyfus wrote: Martin Husemann mar...@duskware.de wrote: However, for a dummie like me who hasn't seen the code in question, could you please explain how they handle the stack pointer in the context? - get current ucontext_t wht

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Joerg Sonnenberger
On Sat, Aug 11, 2012 at 03:37:05AM +0200, Emmanuel Dreyfus wrote: Joerg Sonnenberger jo...@britannica.bec.de wrote: I don't agree with it being a nasty bug. Heck, document it as limitation if you want. But essentially don't mix *context and pthread in this way, it will create other

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Mouse mo...@rodents-montreal.org wrote: Assuming we go with the no spec stance, I think the truth would be more like trying to use contexts across thread boundaries is inherently MD, working the way you expect on ports $LIST1, working this other way on ports $LIST2, and this third way on

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: Which begs the question why they (glusterfs) aren't using something like libcoro in first place... It would be nice from you to subscribe to gluster-de...@nongnu.org and ask there, that would open an insightful discussion. -- Emmanuel Dreyfus

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread matthew sporleder
As an aside- I've recently start using gluster and it's very very cool. I'm really pleased with this porting effort. Thanks manu@!

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Christos Zoulas
On Aug 11, 12:40pm, m...@netbsd.org (Emmanuel Dreyfus) wrote: -- Subject: Re: [RFC][PATCH] _UC_TLSBASE for all ports | Christos Zoulas chris...@zoulas.com wrote: | | I could believe that, if the change suggested to make this the default | behavior (which some would argue it should

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
matthew sporleder msporle...@gmail.com wrote: As an aside- I've recently start using gluster and it's very very cool. On what netbsd and glusterfs versions are you using it? -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Christos Zoulas chris...@zoulas.com wrote: Well, why don't we make it that way then? We cannot toggle an option that does not exist, so that require adding _UC_TLSBASE for ports that miss it. This meets a strong opposition for now. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Christos Zoulas
On Aug 11, 5:13pm, m...@netbsd.org (Emmanuel Dreyfus) wrote: -- Subject: Re: [RFC][PATCH] _UC_TLSBASE for all ports | Well, why don't we make it that way then? | | We cannot toggle an option that does not exist, so that require adding | _UC_TLSBASE for ports that miss it. This meets a strong

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Thor Lancelot Simon
On Sat, Aug 11, 2012 at 06:45:12AM +, Christos Zoulas wrote: It is a slippery slope, but I think in this case it is wise to bend. If we cannot reach agreement here, consult core. I see no point bending NetBSD into knots in this case if the resulting performance is as bad as Joerg claims

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Thor Lancelot Simon
On Sat, Aug 11, 2012 at 03:36:28PM +0200, Emmanuel Dreyfus wrote: Then there is the problem that no spec tells us what swapcontext() should do with pthread private context, as returned by pthread_self(). Gee, I wonder why this stuff was removed from the standard. Thor

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Thor Lancelot Simon t...@panix.com wrote: Gee, I wonder why this stuff was removed from the standard. It seems the biggest problem is that makecontext does not specify argument types. If you send a pointer on an LP64 machine and the implementation assumes arguments are int, you are screwed. --

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Joerg Sonnenberger
On Sat, Aug 11, 2012 at 01:35:43PM -0400, Thor Lancelot Simon wrote: On Sat, Aug 11, 2012 at 06:45:12AM +, Christos Zoulas wrote: It is a slippery slope, but I think in this case it is wise to bend. If we cannot reach agreement here, consult core. I see no point bending NetBSD into

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: Yes, I dislike it being used, because it breaks perfectly valid assumptions like lwpid being constant across the life time of a thread. Therefore I do not understand why you oppose the proposal of proposing a MI interface to let the user opt

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Joerg Sonnenberger
On Sat, Aug 11, 2012 at 03:42:59PM +0200, Emmanuel Dreyfus wrote: Joerg Sonnenberger jo...@britannica.bec.de wrote: Yes, I dislike it being used, because it breaks perfectly valid assumptions like lwpid being constant across the life time of a thread. Therefore I do not understand why

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Matt Thomas
On Aug 11, 2012, at 10:35 AM, Thor Lancelot Simon wrote: On Sat, Aug 11, 2012 at 06:45:12AM +, Christos Zoulas wrote: It is a slippery slope, but I think in this case it is wise to bend. If we cannot reach agreement here, consult core. I see no point bending NetBSD into knots in this

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
matthew green m...@eterna.com.au wrote: i'm confused how the _UC_* interface can be considered public. the whole point of them being _* is that they're private to the implementation, and depending on them is wrong. Yes, _UC_* is out of the standard since there is no uc_field:

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Valeriy E. Ushakov
On Sat, Aug 11, 2012 at 16:03:08 -0700, Matt Thomas wrote: On Aug 11, 2012, at 10:35 AM, Thor Lancelot Simon wrote: On Sat, Aug 11, 2012 at 06:45:12AM +, Christos Zoulas wrote: It is a slippery slope, but I think in this case it is wise to bend. If we cannot reach agreement here,

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Emmanuel Dreyfus
Emmanuel Dreyfus m...@netbsd.org wrote: For now if you want to swapcontext() between threads, you can do that before calling swapcontext() on amd64, arm, hppa, i386, m68k and vax: ctx.uc_flags = _UC_TLSBASE I meant this (missing ~): ctx.uc_flags = ~_UC_TLSBASE -- Emmanuel

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-11 Thread Christos Zoulas
On Aug 11, 1:35pm, t...@panix.com (Thor Lancelot Simon) wrote: -- Subject: Re: [RFC][PATCH] _UC_TLSBASE for all ports | On Sat, Aug 11, 2012 at 06:45:12AM +, Christos Zoulas wrote: | | It is a slippery slope, but I think in this case it is wise to bend. | If we cannot reach agreement

[RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Emmanuel Dreyfus
Hi all Summary of the previous episode: NetBSD's swapcontext restores the thread_self pointer. When using swapcontext() on a context obtained from getcontext() in another thread, this makes two threads with the same pthread_self, leading to chaos Some ports have a workaround for this: unset

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Matt Thomas
On Aug 10, 2012, at 9:58 AM, Joerg Sonnenberger wrote: On Fri, Aug 10, 2012 at 04:36:17PM +, Emmanuel Dreyfus wrote: Summary of the previous episode: NetBSD's swapcontext restores the thread_self pointer. When using swapcontext() on a context obtained from getcontext() in another

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Emmanuel Dreyfus
Matt Thomas m...@3am-software.com wrote: I concur. A thread's stack is its identity and one can't swap stacks without swapping its sense of self. (granted thread local storage helps but that isn't universally available). This is not about the stack, this is about pthread_self() which can be

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Valeriy E. Ushakov
On Fri, Aug 10, 2012 at 10:19:03 -0700, Matt Thomas wrote: On Aug 10, 2012, at 9:58 AM, Joerg Sonnenberger wrote: On Fri, Aug 10, 2012 at 04:36:17PM +, Emmanuel Dreyfus wrote: Summary of the previous episode: NetBSD's swapcontext restores the thread_self pointer. When using

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Matt Thomas
On Aug 10, 2012, at 10:32 AM, Emmanuel Dreyfus wrote: Matt Thomas m...@3am-software.com wrote: I concur. A thread's stack is its identity and one can't swap stacks without swapping its sense of self. (granted thread local storage helps but that isn't universally available). This is

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Emmanuel Dreyfus
Matt Thomas m...@3am-software.com wrote: if you start using using another thread's stack, you are assuming the identity of that thread. It is some part of its identity, but not all. Another important part of the thread identity is given by pthread_self(). That is used to ensure mutexes, locks

Re: [RFC][PATCH] _UC_TLSBASE for all ports]

2012-08-10 Thread Joerg Sonnenberger
On Fri, Aug 10, 2012 at 08:00:32PM +0200, Emmanuel Dreyfus wrote: I just propose to add an option to avoid that. I have trouble to understand why it meets such a resistance, since most of our ports already have that option available. I just add missing bits. The only reason the option exists

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread David Holland
On Fri, Aug 10, 2012 at 08:00:32PM +0200, Emmanuel Dreyfus wrote: if you start using using another thread's stack, you are assuming the identity of that thread. It is some part of its identity, but not all. [...] I just propose to add an option to avoid that. I have trouble to

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Emmanuel Dreyfus
David Holland dholland-t...@netbsd.org wrote: Only by addressing the complete picture can we reasonably expect to decide on a semantically sound interface that we won't regret in the future. Short reply, I will address the rest later: We already have the _UC_TLSBASE feature for most of our

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Emmanuel Dreyfus
Mouse mo...@rodents-montreal.org wrote: Or are we talking a strictly cooperative multitasking model in which a thread's stack must be empty (or at least at some fixed depth) when it yields? Yes, this is used for cooperative multitasking of N user threads over P kernel threads. -- Emmanuel

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: I maintain that trying to move contexts between threads is an inherently bad idea and that it is a very inefficient interface for implementing coroutines. I object to this change for the sake of misdesigned software. Did you look at that test

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Joerg Sonnenberger
On Fri, Aug 10, 2012 at 07:31:59PM +0200, Emmanuel Dreyfus wrote: Joerg Sonnenberger jo...@britannica.bec.de wrote: I maintain that trying to move contexts between threads is an inherently bad idea and that it is a very inefficient interface for implementing coroutines. I object to this

Re: [RFC][PATCH] _UC_TLSBASE for all ports

2012-08-10 Thread Emmanuel Dreyfus
Joerg Sonnenberger jo...@britannica.bec.de wrote: I don't agree with it being a nasty bug. Heck, document it as limitation if you want. But essentially don't mix *context and pthread in this way, it will create other interesting issues later. Ok, so here is my point of view: - we allow mixing