Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Fri, 1 Feb 2019 11:50:58 -0500 Andrew Piskorski wrote: > work around the "Nsvs only store strings" limitation This "limitation" is rather the only generic and safe option! And it is pretty trivial in implementation. To see the other possibility, look in the tsv in the threading extension. It is much more complicated but allows one to "share" objects between threads w/o constant to/from string conversion. But it is black magic (rather complex) and limited (by the very nature of Tcl_Objs) to some well known simple object types. Cheers, Zoran ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
On Fri, 1 Feb 2019 11:15:46 -0500 Andrew Piskorski wrote: > since I'm not using virtual servers anyway, how do I say > "all of them, process-wide"? You can't. The nsv buckets are stored on per-server basis. Naviserver may run a number of virtual servers. You declare all this in the configuration file. See "ns/servers" part in your configuration file. It is your module that, when loaded in the Naviserver, gets the name of the current server it is loaded into. To be loaded as extension, your code needs to provide Ns_ModuleInit() call that gets called by the module loader when a virtual server is getting initialized. The system passes the name of the server and name of the module to your Ns_ModuleInt(). You need to store it somewhere for later use. HTH, ZOran ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Fri, Feb 01, 2019 at 05:28:50PM +0100, Zoran Vasiljevic wrote: > OTOH, if you use ns:addr object, you need not take care about > locking and all the other trivia at all. It is the "rest" of > the system that needs to do that. > > Some of the stuff in "nsd/tclobj.c" seems be primarily concerned with > > the "address object type" (IP addresses?), but I get the sense that > It's (memory-)addresses == pointers. Ah! And you've already written the code to correctly convert between the pointer and string representations, so the ns:addr object approach is also a way to work around the "Nsvs only store strings" limitation. Thanks you, Zoran. I'm not sure which way I'll go in my implementation, but the choices available to me are making more sense now. -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Fri, 1 Feb 2019 10:57:00 -0500 Andrew Piskorski wrote: > What if you don't care much about Tcl objects? A C pointer is already > easily shareable across threads (you just need the proper mutex > locking when using it), the trick is finding it again using some > handle from Tcl. For simplicity, should I just completely ignore the > whole Tcl_Obj machinery when doing that? Of course. You can always use a lock, hash-table and key your pointers, passing the key to the Tcl. But you asked for some other way to do it :-) > > Or SHOULD I (always?) make my handle into a Tcl_Obj? If so, can you > go into a bit more detail on WHY that's a better approach? Even > despite the fact that this Tcl_Obj thing is now more difficult to > share across threads? By having a Tcl object that contains ptr rep, you can safely toss it arround Tcl as it will be a "natural" Tcl thing. You can treat it everywhere as a regular Tcl object. Whereas if you opt for the above approach, you need to "know" what that key points to. If you are only consumer of that key it is pretty OK to do it. But if you are passing ptr's arround the "unknown" code you might be better with object. This is why we did the ns:addr object, so that we do not have to care. It is really the same thing but other way around. > > If I take an ordinary sort of Tcl string or integer and hand it to my > C code, that works fine as the key for an nsv or Tcl_HashTable lookup. > Of course, the code doesn't "know" that that particular string is > supposed to be a handle, but there's nothing stopping me from simply > using it as one in the C code, and throwing an error if I can't find a > hash entry for it. Exactly. > > I suspect that the above is NOT the recommended approach, but I don't > fully understand why. It seems "simpler" to me... But maybe that's > only because I don't yet understand the Tcl_Obj integration approach, > and what benefits it provides. It is "simpler" because you are used to it. But in essence, you need to make a shared hash table, invent a key for your pointers and make sure all is always properly locked. So it isn't per-se simpler. It is still a devil, but that devil isn't that black because you already know it :-) OTOH, if you use ns:addr object, you need not take care about locking and all the other trivia at all. It is the "rest" of the system that needs to do that. > Hm, that has Ns_TclSetTwoPtrValue(), Ns_TclGetOpaqueFromObj(), and > other black magic (or so it seems to me). Are there docs on these C > APIs anywhere? No. But it is really trivial. If you want to wrap a ptr into the ns:addr object you can do: Tcl_Obj *myObj = Tcl_NewObj(); char *myType = "at-ptr"; /* Name it as you want */ MyDataStruct *myStruct = &MyGlobalStruct; Ns_TclSetAddrObj(myObj, myType, (void*)myStruct); Now you can pass the myObj to Tcl (put it in result, bury it in some list, store it in nsv array etc.pp.). Your code may later access the address/ptr value: Tcl_Obj *myObj = objv[1]; /* For example */ MyDataStruct *myStruct; Ns_TclGetOpaqueFromObj(myObj, myType, (void**)&myStruct); so now you have (or don't if GetOpaque retuns TCL_ERROR) your initial *myStruct value. > > Some of the stuff in "nsd/tclobj.c" seems be primarily concerned with > the "address object type" (IP addresses?), but I get the sense that > much of that code is much more general purpose than that. I see that > "nsd/tcltime.c" uses the tclobj.c functions to implement an Ns_Time > Tcl_Obj type, but I definitely don't understand how or why yet! > It's (memory-)addresses == pointers. HTH, Zoran ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
On Fri, Feb 01, 2019 at 04:25:20PM +0100, Zoran Vasiljevic wrote: > > I'm a little confused by (e.g.) Ns_VarGet(), though. Why is its first > > arg a string for the "server"? What does that mean, and where am I > > supposed to get it from? > > This is the identification of the Naviserver's virtual server > that is implicitly "known" to the Tcl code. The nsv_XXX is > thus bound to a virtual server since your Tcl code executes > within a specific virtual server. But on the C-level you need > to specify the virtual server explicitly because your C code > may operate for all known virtual servers or just bunch of them. Ok. But where do I get the server string from, in C? Or since I'm not using virtual servers anyway, how do I say "all of them, process-wide"? -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Fri, Feb 01, 2019 at 11:44:45AM +0100, Zoran Vasiljevic wrote: > The only safe way to share data between threads > in Tcl is over strings. One can go to some extent > and try making Tcl objects shareable but for that > you need to know the object internals which is a > great PITA and does not always work well, except for > a limited set of object types. What if you don't care much about Tcl objects? A C pointer is already easily shareable across threads (you just need the proper mutex locking when using it), the trick is finding it again using some handle from Tcl. For simplicity, should I just completely ignore the whole Tcl_Obj machinery when doing that? Or SHOULD I (always?) make my handle into a Tcl_Obj? If so, can you go into a bit more detail on WHY that's a better approach? Even despite the fact that this Tcl_Obj thing is now more difficult to share across threads? If I take an ordinary sort of Tcl string or integer and hand it to my C code, that works fine as the key for an nsv or Tcl_HashTable lookup. Of course, the code doesn't "know" that that particular string is supposed to be a handle, but there's nothing stopping me from simply using it as one in the C code, and throwing an error if I can't find a hash entry for it. I suspect that the above is NOT the recommended approach, but I don't fully understand why. It seems "simpler" to me... But maybe that's only because I don't yet understand the Tcl_Obj integration approach, and what benefits it provides. On Fri, Feb 01, 2019 at 10:29:47AM +0100, Zoran Vasiljevic wrote: > You might want to look into: > >nsd/tclobj.c Hm, that has Ns_TclSetTwoPtrValue(), Ns_TclGetOpaqueFromObj(), and other black magic (or so it seems to me). Are there docs on these C APIs anywhere? Some of the stuff in "nsd/tclobj.c" seems be primarily concerned with the "address object type" (IP addresses?), but I get the sense that much of that code is much more general purpose than that. I see that "nsd/tcltime.c" uses the tclobj.c functions to implement an Ns_Time Tcl_Obj type, but I definitely don't understand how or why yet! -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
On Fri, 1 Feb 2019 09:59:09 -0500 Andrew Piskorski wrote: > So they do the exact same thing, and are completely interoperable? > Meaning I can have C and Tcl talk to each other via Ns_Var* in C and > nsv_* in Tcl? Great! Yes. > > I'm a little confused by (e.g.) Ns_VarGet(), though. Why is its first > arg a string for the "server"? What does that mean, and where am I > supposed to get it from? This is the identification of the Naviserver's virtual server that is implicitly "known" to the Tcl code. The nsv_XXX is thus bound to a virtual server since your Tcl code executes within a specific virtual server. But on the C-level you need to specify the virtual server explicitly because your C code may operate for all known virtual servers or just bunch of them. > > Also, why are NsTclNsvGetObjCmd() and Ns_VarGet() > implemented independently, rather than NsTclNsvGetObjCmd() calling > Ns_VarGet()? I assume there's some good reason for this, but I'd like > to understand what it is... Well, simplicity, that is. In essence, one could have done that but it would be more complicated. The former operates mainly with Tcl objects from within one Tcl script whereas the latter is mainly (supposed to be) used from C code that has no relation to any Tcl interp/script. If you compare the guts of both calls you will see something like LockArray/CreateHashEntry which is what "counts". Everything else is glue. HTH, Zoran ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
On Fri, Feb 01, 2019 at 10:22:46AM +0100, Zoran Vasiljevic wrote: > Andrew Piskorski wrote: > > > I know that the Tcl nsv_get command is implemented by > > NsTclNsvGetObjCmd(). But in the NaviServer C code, what is > > Ns_VarGet() for? I don't see it used anywhere. Is Ns_VarGet() a C > > interface to exact same nsv_get functionality, or is it for something > > completely different? > > The Ns_VarXXX is a programatic interface (from C) > to the nsv (Naviserver Shared Variables) storage. So they do the exact same thing, and are completely interoperable? Meaning I can have C and Tcl talk to each other via Ns_Var* in C and nsv_* in Tcl? Great! I'm a little confused by (e.g.) Ns_VarGet(), though. Why is its first arg a string for the "server"? What does that mean, and where am I supposed to get it from? Also, why are NsTclNsvGetObjCmd() and Ns_VarGet() implemented independently, rather than NsTclNsvGetObjCmd() calling Ns_VarGet()? I assume there's some good reason for this, but I'd like to understand what it is... -- Andrew Piskorski ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Thu, 31 Jan 2019 15:46:59 -0500 Andrew Piskorski wrote: > And I already know how to call > NsTclNsvGetObjCmd() etc. from C. But the Nsv functions all operate > ONLY on strings... Oh yes, never use NsXXX calls in your code as these are not exported and may change anytime. Always use Ns_XXX instead. The only safe way to share data between threads in Tcl is over strings. One can go to some extent and try making Tcl objects shareable but for that you need to know the object internals which is a great PITA and does not always work well, except for a limited set of object types. Cheers, Zoran ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] store C pointers in Nsv vs. Tcl_HashTable?
On Thu, 31 Jan 2019 15:46:59 -0500 Andrew Piskorski wrote: > In general, what's the recommended way for Tcl and/or NaviServer > programmers to allocate handles to C-level pointers, and then access > those handles and pointers from more than one Tcl thread? Can anyone > point me to examples of code doing it the right way? You might want to look into: nsd/tclobj.c Cheers, Zoran ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
Re: [naviserver-devel] Ns_VarGet vs. NsTclNsvGetObjCmd
On Thu, 31 Jan 2019 14:48:11 -0500 Andrew Piskorski wrote: > I know that the Tcl nsv_get command is implemented by > NsTclNsvGetObjCmd(). But in the NaviServer C code, what is > Ns_VarGet() for? I don't see it used anywhere. Is Ns_VarGet() a C > interface to exact same nsv_get functionality, or is it for something > completely different? The Ns_VarXXX is a programatic interface (from C) to the nsv (Naviserver Shared Variables) storage. Cheers, Zoran ___ naviserver-devel mailing list naviserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/naviserver-devel
