Re: [zeromq-dev] Questions about Coding Style
On 13/02/2012, at 12:29 PM, Chuck Remes wrote: > I think allowing bindings to go directly to the C++ core interface will be > problematic for a lot of languages. Many languages interface to libzmq via > FFI which, as far as I know, doesn't support C++ in any way, shape or form. > These languages would still need to build a binding via the C API. > > Correct me if I'm wrong (which I may be). I'm sure you're right. If there an API for bindings, we might well need a C one. And perhaps a C++ one as well. Of course, a binding can still use the Posix/C API: they do that now. BTW: this is a thought experiment at the moment. Having two APIs would suck in many ways. Double maintenance, quadruple confusion .. :) -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 12, 2012, at 5:01 PM, john skaller wrote: > > On 13/02/2012, at 9:24 AM, Gary Wright wrote: >> >> Perhaps the solution is to have two interfaces to the library. > > Heh! I just wrote that too. I think allowing bindings to go directly to the C++ core interface will be problematic for a lot of languages. Many languages interface to libzmq via FFI which, as far as I know, doesn't support C++ in any way, shape or form. These languages would still need to build a binding via the C API. Correct me if I'm wrong (which I may be). cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 13/02/2012, at 9:24 AM, Gary Wright wrote: > > Perhaps the solution is to have two interfaces to the library. Heh! I just wrote that too. > A > 'paranoid' interface that reports errors as much as possible and a > 'go wild' interface that simply assumes that everything has been > checked. The 'paranoid' interface can simply wrap the 'go wild' > interface and clients can pick and choose the interface that meets > their needs? It isn't just a matter of checking. The interface a C application programmer needs is different to what a binding needs. For example, if zmq_msg is useful at all in C++ (which I have doubts about), it needs to have default_init // cannot fail, returns void reset // close then init init_data/size // reset then do the init_data/size It isn't safe to close a non-init'd object, but this just can't happen in C++ if the constructors call default_init, unless the user does something weird with casts or whatever (which will break anything you want). And we don't want to call msg_move or copy: C++ has assignment, copy constructors, and in C++xx a move operation. The only way to make this work cleanly in C++ with the current C interface and semantics is to add a state flag to the wrapper object. The thing is .. the C API is already a wrapper around C++ :) In some languages (like Felix) the destructor could be executed off-line by a garbage collector, so the close has to either work or kill the process. It's the same in C++ except there you have exceptions, and it's possible the destructor is being executed due to one being thrown. Again, you want to release any resources but you cannot permit failure (or you get a double fault). > I think this is an interesting discussion, but it is also an abstract > one about API design principles. I don't pretend to have an informed > opinion on how this might best be applied to the ZMQ code base. It's not that hard. The existing C API is intended for standardisation and use by C application programmers. It has to be left alone pretty much. That doesn't prevent a second API being created for language binders. Actually, there already is one (the core C++ code). After all that's how the C binding works. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 13/02/2012, at 8:28 AM, Gary Wright wrote: > ruby client code => ruby binding => libzmq C binding => C++ core I actually think this is a problem. Basically, the C binding is just that: it's just another binding. What I mean is, there should probably be two interfaces: 1) Core interface, to be used by bindings 2) C/Posix binding, to be used by C application programmers -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 12, 2012, at 4:40 PM, Mikko Koppanen wrote: > On Sun, Feb 12, 2012 at 9:28 PM, Gary Wright wrote: >> ruby client code => ruby binding => libzmq C binding => C++ core >> >> So I would just apply the principle described above to each interface. >> >> The C++ code should use error codes/exceptions according to C++ idioms. >> The libzmq C interface should translate C++ exceptions into error codes. >> A Ruby binding, for example, should translate some errors back into >> exceptions. > > Hello, > > I don't necessarily agree that the library code should take care of > validating every argument passed to it. I don't think I actually said that. In fact I agreed with the fact that violation of pre-conditions is a problem with the caller, not the callee. I was trying to acknowledge both sides of the argument. > In your example the ruby binding should take care of the sanity of the > arguments passed to lower level or the ruby binding can be built on > top of czmq, which does more validation. I don't disagree with this in principle, but libzmq, for example, isn't *only* used by other libraries (i.e., language binding). It is possible to use it directly and so we find ourself having the exact same decision to be made at that level (i.e. check the arguments or not). Perhaps the solution is to have two interfaces to the library. A 'paranoid' interface that reports errors as much as possible and a 'go wild' interface that simply assumes that everything has been checked. The 'paranoid' interface can simply wrap the 'go wild' interface and clients can pick and choose the interface that meets their needs? > Adding more validation and cruft on the critical path means additional > CPU cycles wasted, which is not desirable in all cases. In my opinion > the raw library can and should assume that sane arguments are being > passed in and the higher level interfaces (czmq for example) can and > should do the validation on top of it if necessary. I sympathize with that point of view and perhaps czmq is the 'paranoid' library in this case? I think this is an interesting discussion, but it is also an abstract one about API design principles. I don't pretend to have an informed opinion on how this might best be applied to the ZMQ code base. I mainly jumped in to express some concern with the idea that assert/abort/exit is anything other than absolutely last course of action in the context of library code. Gary Wright ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Sun, Feb 12, 2012 at 9:28 PM, Gary Wright wrote: > ruby client code => ruby binding => libzmq C binding => C++ core > > So I would just apply the principle described above to each interface. > > The C++ code should use error codes/exceptions according to C++ idioms. > The libzmq C interface should translate C++ exceptions into error codes. > A Ruby binding, for example, should translate some errors back into > exceptions. Hello, I don't necessarily agree that the library code should take care of validating every argument passed to it. In your example the ruby binding should take care of the sanity of the arguments passed to lower level or the ruby binding can be built on top of czmq, which does more validation. Adding more validation and cruft on the critical path means additional CPU cycles wasted, which is not desirable in all cases. In my opinion the raw library can and should assume that sane arguments are being passed in and the higher level interfaces (czmq for example) can and should do the validation on top of it if necessary. -- Mikko ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 12, 2012, at 1:39 AM, john skaller wrote: > > For example, in the C binding, a C++ "null pointer exception" thrown could be > caught and translated into something else, eg EINVAL return code. > It seems risky: ZMQ specifies in the Coding Style that no exceptions should > be thrown. This probably means the code doesn't try to be exception safe. > In turn that means returning an error code is risky, even for a language > binding. > > Gary, you have an actual use case? Note sure what you mean by 'use' case. As a general principle I don't think a library should abort or otherwise terminate a process unless there is no safe way for the error to be reported up the call stack. Errors should be reported back to the client code in some reasonable, hopefully idiomatic way. For example, in Ruby, it is common for invalid arguments to be flagged by raising the ArgumentError exception (wrong number of arguments, invalid argument, etc.) I do tend to agree with the idea that if the library client code violates a pre-condition then all bets are off but I also realize that detecting those violations in the library can be helpful during debugging and can prevent more obscure problems from occurring later in execution. In the ZMQ context this is all complicated by the fact that the core code is written in C++, while most language bindings (I believe) tend to build upon a C wrapper for the C++ core. So you've something like: ruby client code => ruby binding => libzmq C binding => C++ core So I would just apply the principle described above to each interface. The C++ code should use error codes/exceptions according to C++ idioms. The libzmq C interface should translate C++ exceptions into error codes. A Ruby binding, for example, should translate some errors back into exceptions. Gary Wright ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 12/02/2012, at 11:40 AM, Gary Wright wrote: > > On Feb 11, 2012, at 4:53 PM, Pieter Hintjens wrote: >> >> So in my libraries, like CZMQ, I validate input arguments with >> asserts, systematically. > > And yet for some language bindings it would be more idiomatic to > raise an exception, rather than just have the process terminate. If > libzmq returned an error code, the language binding could 'do > the right thing'. That's a good point. Actually asserts() are used for two distinct purposes: (1) a bug in ZMQ (2) a bug in the parameters or environment (a) null pointer (b) memory corruption detected There may be more, eg out of memory, I don't know. A bug in ZMQ should crash the process. The "right thing" is not really so important (other than providing some help to find the bug). However the other cases might warrant an exception, especially in a long running server which just sloppy validating a client module .. you may want to kill the module, but not the server. It's not clear an error handler would actually allow an immediate return with an error code, but it *could* throw an exception. [The assert could be buried deeply in function calls that can't return error codes] And the a binding could catch it and then do what is required. For example, in the C binding, a C++ "null pointer exception" thrown could be caught and translated into something else, eg EINVAL return code. It seems risky: ZMQ specifies in the Coding Style that no exceptions should be thrown. This probably means the code doesn't try to be exception safe. In turn that means returning an error code is risky, even for a language binding. Gary, you have an actual use case? -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 11, 2012, at 4:53 PM, Pieter Hintjens wrote: > > So in my libraries, like CZMQ, I validate input arguments with > asserts, systematically. And yet for some language bindings it would be more idiomatic to raise an exception, rather than just have the process terminate. If libzmq returned an error code, the language binding could 'do the right thing'. Gary Wright ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 11, 2012, at 3:15 PM, john skaller wrote: > > On 12/02/2012, at 7:44 AM, Chuck Remes wrote: >> >> John, why do you make this more complex than it needs to be? > > I'm making a point. There are alternate viewpoints, > and compromises in between. Both viewpoints are valid, > as are compromises in between. I know, but I don't think it adds much to the conversation. You "make a point" in *every* thread on this list. It's getting tiring. >> In other words, quit being some fucking pedantic. :) so you know I don't >> actually hate you > > I like being pedantic :) You make a federal case out of everything even when the issue is minor. Again, it's tiring. I'll likely not participate in future threads where you do this. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Sun, Feb 12, 2012 at 4:47 AM, niXman wrote: > Do you all agree with that the arguments must be checked? And with the > fact that in case of error of arguments is not necessary to use assert > but set errno and return -1? Up to a point. Some invalid arguments are best handled with an assertion. The question is, can the calling program do anything useful with a return code? If it is passing a buggy argument, say a NULL socket, it is much safer to assert than to pass a return code. Buggy code is not trustable. I.e. good chance it won't properly handle the return code. However if a program gets input from, say, a socket, it cannot assert if that input is bad. The question is, "is error handling part of the core functionality of this path, or not". If it is, then check everything pedantically. If not, asserts are safer and will lead to more robust code, faster. You can easily prove this by taking a common mistake, e.g. passing an uninitialized message to a send() call. Now, compare using a -1 return code to asserting. The return code requires more code in the application (which means more chance of further errors), and creates more chance that error won't be caught at all, leading to strange behavior that is hard to debug. Whereas an assert is immediately caught, at the precise place the error happens, and it tells the user precisely what is wrong. So in my libraries, like CZMQ, I validate input arguments with asserts, systematically. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 12/02/2012, at 8:03 AM, niXman wrote: >> >> Sort of. You don't need to actually write an allocator. >> It is enough to trace all constructed sockets. > > And what to do with pointers to the data for zmq_recv (), for example? > > void *data = 0x3423e65; > zmq_recv(sock, data, 32, 0); Of course, the logging stuff only catches errors with sockets. > Regarding to the effeciency of malloc (): It is really very effective, > but only for memory allocation. malloc () can not prevent the > fragmentation of memory for the same reasons as the filesystem drivers > can not prevent the fragmentation of the filesystem. > > For systems 24/7, the fragmentation of memory is a very big problem. I > know whereof I speak. Yeah, I know .. many hours spent running defrag on DOS disks :) Your problem is that 0MQ is a library, not a framework. If you want to do a better allocator that's used language wide, come over to the Felix project where most of the allocations are already done through a hook and there's already a way to replace it. Judy uses a hookable allocator, but that's only for internal memory. The case above, arguments to zmq_recv, wouldn't be covered by that. OTOH, allocations to zmq_msg_t might be covered if you changed the protocol. Perhaps I'm missing something. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 12/02/2012, at 7:44 AM, Chuck Remes wrote: > On Feb 11, 2012, at 2:14 PM, john skaller wrote: > >> >> On 12/02/2012, at 4:37 AM, Chuck Remes wrote: >> >>> I will aggressively merge patches that make 0mq a safer and more >>> "forgiving" library. >> >> No you won't :) >> >> You want a robust check? I can give you 99.%. >> But you'll pay. > > John, why do you make this more complex than it needs to be? I'm making a point. There are alternate viewpoints, and compromises in between. Both viewpoints are valid, as are compromises in between. The OP is claiming not making checks is laziness and gives C a bad name, I'm countering that. In fact fragile programming is supported by the contract programming paradigm, and the idea of one point of responsibility. The point is .. there's a value judgement to be made. In a community like this, where there's no easy way to make a judgement, I at least would just follow the precedent if unsure. > I don't need 99.%. I'm happy with less and I bet others are too. If you use a decent programming language you get used to 100%. I'm used to that. I've been doing most of my programming in Ocaml for some time. It's a bit hard facing the reality of using crud like C again. I have to face that in my own project though, because it's intended to be a better C, and so has to bind to C. I'm a masochist :) The problem is what to do in C. What the library does now, check for a signature, is pretty good statistically. It's almost as good as logging for pointer validity and catches a good fraction of corruptions as well, which logging doesn't, and it's also reasonably cheap given that most of the socket related calls are going to be quite expensive underneath, unless you're buffering and writing single characters or something. I actually found a couple of the C API socket calls didn't have the check and put it in for consistency. > In other words, quit being some fucking pedantic. :) so you know I don't > actually hate you I like being pedantic :) There's actually a serious question here: whether to do more intensive and expensive checks, and conditionally compile them out. Assert checks are already conditionally compiled away by -DNDEBUG. In Felix, assertions CANNOT be compiled away; they're retained even in production code. I made that choice because bugs in production code are extremely expensive (client sues vendor .. or give product a bad name) especially if it is almost impossible to replace the library with a debugging version (particularly over the phone :) So again, there's a value judgement to be made. I can put in logging: it's easy enough to add a STL Set to a context and add each socket created to the set, and lock accesses to it, then, you conditionally compile it all away for production libraries. But it isn't clear it is worthwhile. A similar issue arises with fixing the bad type used for sockets. Void * is just a woeful choice. Using a proper type will catch a lot or errors for free at compile time, much better than run time. But it will break some code. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
2012/2/12 john skaller : > > On 12/02/2012, at 7:21 AM, niXman wrote: > >> The problem with dangling pointers can only be solved using its own >> allocator. > > Sort of. You don't need to actually write an allocator. > It is enough to trace all constructed sockets. And what to do with pointers to the data for zmq_recv (), for example? void *data = 0x3423e65; zmq_recv(sock, data, 32, 0); Regarding to the effeciency of malloc (): It is really very effective, but only for memory allocation. malloc () can not prevent the fragmentation of memory for the same reasons as the filesystem drivers can not prevent the fragmentation of the filesystem. For systems 24/7, the fragmentation of memory is a very big problem. I know whereof I speak. > > This won't stop memory corruption of a legitimately constructed > live socket. Neither will a custom allocator. > > However I take the view that malloc is already pretty good. > I don't know if it actually is, but it is managed by people that > care a lot about efficiency and probably already provide > checks (on my Mac I actually get "free pointer not allocated" > indicating some checks are being done already). > > If you want a fast allocator, you have to use TLS to avoid locks. > You're allocating excess storage. If your application is using a lot > of libraries all with their own allocators .. you're actually *causing* > fragmentation and wasting memory. > > A library cannot replace "malloc", only a framework should do that. > libgc does it, but then it's really a framework. > > But I'm guessing .. always worth looking at a design proposal. > > > -- > john skaller > skal...@users.sourceforge.net > > > > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 11, 2012, at 2:14 PM, john skaller wrote: > > On 12/02/2012, at 4:37 AM, Chuck Remes wrote: > >> I will aggressively merge patches that make 0mq a safer and more "forgiving" >> library. > > No you won't :) > > You want a robust check? I can give you 99.%. > But you'll pay. John, why do you make this more complex than it needs to be? I don't need 99.%. I'm happy with less and I bet others are too. If someone produces patches that add robustness without a lot of cost (performance or code complexity) then we're all better off. If you want to prove a point and ruin the library with crazy and/or pointless checks to get to 99.% robustness, I bet those patches will get reverted very quickly. In other words, quit being some fucking pedantic. :) so you know I don't actually hate you cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 11, 2012, at 1:47 PM, niXman wrote: > I'm confused ... > Do you all agree with that the arguments must be checked? And with the > fact that in case of error of arguments is not necessary to use assert > but set errno and return -1? I think this is reasonable. If the community doesn't like your patches, they'll get reverted. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 12/02/2012, at 7:21 AM, niXman wrote: > The problem with dangling pointers can only be solved using its own > allocator. Sort of. You don't need to actually write an allocator. It is enough to trace all constructed sockets. This won't stop memory corruption of a legitimately constructed live socket. Neither will a custom allocator. However I take the view that malloc is already pretty good. I don't know if it actually is, but it is managed by people that care a lot about efficiency and probably already provide checks (on my Mac I actually get "free pointer not allocated" indicating some checks are being done already). If you want a fast allocator, you have to use TLS to avoid locks. You're allocating excess storage. If your application is using a lot of libraries all with their own allocators .. you're actually *causing* fragmentation and wasting memory. A library cannot replace "malloc", only a framework should do that. libgc does it, but then it's really a framework. But I'm guessing .. always worth looking at a design proposal. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 12/02/2012, at 6:47 AM, niXman wrote: > I'm confused ... > Do you all agree with that the arguments must be checked? And with the > fact that in case of error of arguments is not necessary to use assert > but set errno and return -1? IMHO, if you do a check for a NULL pointer you should use assert. You want to crash the program instantly, not return an error the user might not test. If the check is about resources, eg "is there enough disk space" then you probably return an error code, because that condition is outside the programmers control, but doing something about it inside the program may not be. The one I get a lot is: port address not available. I try to bind a socket, but it is already in use (because there's already another server running). The client program could chose another port. So this one returns an error code. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
The problem with dangling pointers can only be solved using its own allocator. Thus, any address (pointer), not found among the selected blocks - is considered hanging. This, in its turn will not only reduce the number of errors, but that is not unimportant - will increase system reliability and will exclude memory fragmentation, which is extremely important for the systems 24/7. Therefore, in my first email I asked if there are in Community's plans to implement memory-pool/object-pool/message-pool. 2012/2/11 niXman : > I'm confused ... > Do you all agree with that the arguments must be checked? And with the > fact that in case of error of arguments is not necessary to use assert > but set errno and return -1? > > > > 2012/2/11 Pieter Hintjens : >> On Sat, Feb 11, 2012 at 6:27 PM, niXman wrote: >> >>> For example the call: rc = zmq_setsockopt (sub, ZMQ_SUBSCRIBE, 0, 4); >>> will end on segmentation fault on line 56 of the sub.cpp file. >>> >>> According to the documentation, if an error occurs, the function >>> should return -1. >>> >>> I can not decide how to handle such a situation. Should I use assert >>> or return EINVAL? >> >> Assertions were historically misused in ZeroMQ to handle user errors. >> They should be used only to assert impossible conditions, i.e. >> internal errors where continuing is unacceptable. It is (I've used >> this analogy before) like a cell being highly resistant to invading >> bacteria and viruses, but self-destructing it if gets a mutation in >> its DNA. >> >> Any remaining cases of asserting on bad input are considered bugs, and >> can be fixed as such (issue, test case, and patch). >> >> Thanks a lot for the code review, this is great. >> >> -Pieter >> ___ >> zeromq-dev mailing list >> zeromq-dev@lists.zeromq.org >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 12/02/2012, at 4:37 AM, Chuck Remes wrote: > I will aggressively merge patches that make 0mq a safer and more "forgiving" > library. No you won't :) You want a robust check? I can give you 99.%. But you'll pay. You already griped about the tiny weeny cost of my thread safe sockets (two tests per call). You are not going to like a proper check at all *** You may not even like the patch to fix the stupid typing of sockets, which is entirely free (0 run time cost). That will catch most stupid errors, but it will break a lot of code written in C but compiled with C++. *** a proper check in a single threaded application is quite cheap. But for multi-threaded applications, the check requires a lock. It may be worth "conditional compilation" to put the check in, I don't know: the existing check is already reasonable. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 12/02/2012, at 3:53 AM, John D. Mitchell wrote: > Good morning, > > On Feb 11, 2012, at 01:51 , john skaller wrote: >> On 11/02/2012, at 8:27 PM, niXman wrote: > [...] >>> Continuing review of the libzmq code, I have found a few situations >>> when the program will be crushed on segmentation fault. >> >> It's impossible to avoid this in C. > > That's bollocks. Let me rephrase: it is impossible for the library writer to avoid it. Actually, that is bollocks as you say. It really can be done close to 100% but you may not like the cost. > There's certainly a realm of things way out beyond the boundaries that are > beyond the reasonable control of a library like 0mq. Taking care to deal with > basic, fundamental errors of parameters to functions is NOT one of them. > This is a huge, lazy, cop out Sorry: some people favour robust programming. I'm not one of them. You just have to accept that fragile programming advocates have a point too. You may not understand it, but that's no reason to accuse those with a differing view point of being lazy or coping out. It's possible for the library to robustly check the pointers. It doesn't. Why? Because it's expensive. It's a performance hit. The library makes a cursory check .. and it's really stupid because it uses a really bad type for sockets (void* is the worst possible choice). A type based check is free. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
I'm confused ... Do you all agree with that the arguments must be checked? And with the fact that in case of error of arguments is not necessary to use assert but set errno and return -1? 2012/2/11 Pieter Hintjens : > On Sat, Feb 11, 2012 at 6:27 PM, niXman wrote: > >> For example the call: >>>rc = zmq_setsockopt (sub, ZMQ_SUBSCRIBE, 0, 4); >> will end on segmentation fault on line 56 of the sub.cpp file. >> >> According to the documentation, if an error occurs, the function >> should return -1. >> >> I can not decide how to handle such a situation. Should I use assert >> or return EINVAL? > > Assertions were historically misused in ZeroMQ to handle user errors. > They should be used only to assert impossible conditions, i.e. > internal errors where continuing is unacceptable. It is (I've used > this analogy before) like a cell being highly resistant to invading > bacteria and viruses, but self-destructing it if gets a mutation in > its DNA. > > Any remaining cases of asserting on bad input are considered bugs, and > can be fixed as such (issue, test case, and patch). > > Thanks a lot for the code review, this is great. > > -Pieter > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Sat, Feb 11, 2012 at 6:27 PM, niXman wrote: > For example the call: >>rc = zmq_setsockopt (sub, ZMQ_SUBSCRIBE, 0, 4); > will end on segmentation fault on line 56 of the sub.cpp file. > > According to the documentation, if an error occurs, the function > should return -1. > > I can not decide how to handle such a situation. Should I use assert > or return EINVAL? Assertions were historically misused in ZeroMQ to handle user errors. They should be used only to assert impossible conditions, i.e. internal errors where continuing is unacceptable. It is (I've used this analogy before) like a cell being highly resistant to invading bacteria and viruses, but self-destructing it if gets a mutation in its DNA. Any remaining cases of asserting on bad input are considered bugs, and can be fixed as such (issue, test case, and patch). Thanks a lot for the code review, this is great. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Sat, Feb 11, 2012 at 12:37 PM, Chuck Remes wrote: > On Feb 11, 2012, at 10:53 AM, John D. Mitchell wrote: > >> On Feb 11, 2012, at 01:51 , john skaller wrote: >>> On 11/02/2012, at 8:27 PM, niXman wrote: >> [...] Continuing review of the libzmq code, I have found a few situations when the program will be crushed on segmentation fault. >>> >>> It's impossible to avoid this in C. >> >> That's bollocks. >> I consider that programs crash on segmentation fault is an inadmissible error of the ØMQ developers', but not the library user's. >>> >>> No. The right thing is: the library is only responsible if the >>> pre-conditions of the function call are met. If the pre-conditions >>> of the call are not met, all bets are off. >> >> This is *EXACTLY* the attitude that people took who created the various >> libraries and the users who followed their example that has given C this >> particularly bad reputation. >> >> There's certainly a realm of things way out beyond the boundaries that are >> beyond the reasonable control of a library like 0mq. Taking care to deal >> with basic, fundamental errors of parameters to functions is NOT one of >> them. This is a huge, lazy, cop out and, IMHO, seriously hurts the adoption >> of 0mq by people/organizations who need something they can have full faith >> and trust in to run robustly. >> >> This is critically important for the growth of 0mq out beyond it's >> traditional/historical community in the big financials. I.e. out in the >> wilderness where the networks aren't all high-speed and local; where there >> aren't operators on-call 24x7; where all of the other end-points are all >> basically well-behaved; where the programmers aren't living and breathing >> this stuff all day long; etc. >> >> I understand your personal bias to drive people to other, "better" languages >> and applaud your efforts in actually creating a language. But the fact is >> that 0mq is a multi-language solution by design and so there's no hand >> waving away this very fundamental flaw. > > I agree 100% with Mr. Mitchell. > > I will aggressively merge patches that make 0mq a safer and more "forgiving" > library. As a binding author and maintainer, I can't count the number of > times that I made a stupid mistake that my error handling mechanisms would > catch but 0mq puked instead. Validation of arguments is a pretty easy > operation and one that I encourage for libraries. > > I have heard of a "rule" that sounds pretty reasonable for libraries: Be > strict in what you output but tolerant in what you accept as input. Hi all, when it comes to critical systems programming, I simply turn to the Linux kernel developer's coding standards, styles and conventions. Done. just my humble $0.02. ~Peter a.k.a. travlr > > cr > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 11, 2012, at 10:53 AM, John D. Mitchell wrote: > On Feb 11, 2012, at 01:51 , john skaller wrote: >> On 11/02/2012, at 8:27 PM, niXman wrote: > [...] >>> Continuing review of the libzmq code, I have found a few situations >>> when the program will be crushed on segmentation fault. >> >> It's impossible to avoid this in C. > > That's bollocks. > >>> I consider that programs crash on segmentation fault is an >>> inadmissible error of the ØMQ developers', but not the library user's. >> >> No. The right thing is: the library is only responsible if the >> pre-conditions of the function call are met. If the pre-conditions >> of the call are not met, all bets are off. > > This is *EXACTLY* the attitude that people took who created the various > libraries and the users who followed their example that has given C this > particularly bad reputation. > > There's certainly a realm of things way out beyond the boundaries that are > beyond the reasonable control of a library like 0mq. Taking care to deal with > basic, fundamental errors of parameters to functions is NOT one of them. > This is a huge, lazy, cop out and, IMHO, seriously hurts the adoption of 0mq > by people/organizations who need something they can have full faith and trust > in to run robustly. > > This is critically important for the growth of 0mq out beyond it's > traditional/historical community in the big financials. I.e. out in the > wilderness where the networks aren't all high-speed and local; where there > aren't operators on-call 24x7; where all of the other end-points are all > basically well-behaved; where the programmers aren't living and breathing > this stuff all day long; etc. > > I understand your personal bias to drive people to other, "better" languages > and applaud your efforts in actually creating a language. But the fact is > that 0mq is a multi-language solution by design and so there's no hand waving > away this very fundamental flaw. I agree 100% with Mr. Mitchell. I will aggressively merge patches that make 0mq a safer and more "forgiving" library. As a binding author and maintainer, I can't count the number of times that I made a stupid mistake that my error handling mechanisms would catch but 0mq puked instead. Validation of arguments is a pretty easy operation and one that I encourage for libraries. I have heard of a "rule" that sounds pretty reasonable for libraries: Be strict in what you output but tolerant in what you accept as input. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
Good morning, On Feb 11, 2012, at 01:51 , john skaller wrote: > On 11/02/2012, at 8:27 PM, niXman wrote: [...] >> Continuing review of the libzmq code, I have found a few situations >> when the program will be crushed on segmentation fault. > > It's impossible to avoid this in C. That's bollocks. >> I consider that programs crash on segmentation fault is an >> inadmissible error of the ØMQ developers', but not the library user's. > > No. The right thing is: the library is only responsible if the > pre-conditions of the function call are met. If the pre-conditions > of the call are not met, all bets are off. This is *EXACTLY* the attitude that people took who created the various libraries and the users who followed their example that has given C this particularly bad reputation. There's certainly a realm of things way out beyond the boundaries that are beyond the reasonable control of a library like 0mq. Taking care to deal with basic, fundamental errors of parameters to functions is NOT one of them. This is a huge, lazy, cop out and, IMHO, seriously hurts the adoption of 0mq by people/organizations who need something they can have full faith and trust in to run robustly. This is critically important for the growth of 0mq out beyond it's traditional/historical community in the big financials. I.e. out in the wilderness where the networks aren't all high-speed and local; where there aren't operators on-call 24x7; where all of the other end-points are all basically well-behaved; where the programmers aren't living and breathing this stuff all day long; etc. I understand your personal bias to drive people to other, "better" languages and applaud your efforts in actually creating a language. But the fact is that 0mq is a multi-language solution by design and so there's no hand waving away this very fundamental flaw. Take care, John ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
2012/2/11 Chuck Remes : > On Feb 11, 2012, at 5:06 AM, niXman wrote: > >> Before writing something, I want to understand the attitude of the >> project Administrators to innovations, and their readiness to take >> measures for improving libzmq. >> Meanwhile, even my simple patch [1] isn't accepted. >> >> >> [1] https://github.com/zeromq/libzmq/pull/240 > > Merged. Thank you for your contribution to 0mq! Thanks Chuck! > > cr > > ___ > zeromq-dev mailing list > zeromq-dev@lists.zeromq.org > http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Feb 11, 2012, at 5:06 AM, niXman wrote: > Before writing something, I want to understand the attitude of the > project Administrators to innovations, and their readiness to take > measures for improving libzmq. > Meanwhile, even my simple patch [1] isn't accepted. > > > [1] https://github.com/zeromq/libzmq/pull/240 Merged. Thank you for your contribution to 0mq! cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
2012/2/11 john skaller :
>
> On 11/02/2012, at 10:06 PM, niXman wrote:
>
>> Before writing something, I want to understand the attitude of the
>> project Administrators to innovations, and their readiness to take
>> measures for improving libzmq.
>
> It's fairly clear in the policy.
>
>> Meanwhile, even my simple patch [1] isn't accepted.
>>
>> [1] https://github.com/zeromq/libzmq/pull/240
>
> I added a comment. You solution seems reasonable and systematically
> implemented.
>
> The policy more or less requires that it be pulled, so you just need to wait.
>
> The policy here now is more oriented to action rather than talk.
>
> Roughly, if you think something should be done, just do it,
> let people know, if they don't like it they'll say so and you can undo it,
> fix it, or someone else can.
>
> And they will :)
>
> There's no need to fear someone will undo your changes.
> On the contrary, it's a relief!
>
> for example, as I comment on your pull request, in C++ there's another
> way to do it. You chose:
>
> void f(T unused) { (void)unused; .. }
>
> The other way, in C++ only, is:
>
> void f(T) { .. }
>
> i.e. don't name the argument. So, if I feel like it I could change your patch
> to the other way. I won't, the point is, there's a choice, you made it,
> and someone strongly dis-agreeing might re-patch or even revert.
>
> The advantage of that is that you don't have to feel completely responsible.
> Which encourages you to actually make changes.
>
> So basically .. if, after a while, you patch sticks and there's no complaint,
> you should actually put the rule in the Style Guide, how to treat unused
> arguments.
I understand that it OpenSource community. Therefore, I want to
understand relations of Administration of the project to patches and
sentences to add new functionality.
John, thanks for the explanation!
>
> --
> john skaller
> skal...@users.sourceforge.net
>
>
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 11/02/2012, at 10:06 PM, niXman wrote:
> Before writing something, I want to understand the attitude of the
> project Administrators to innovations, and their readiness to take
> measures for improving libzmq.
It's fairly clear in the policy.
> Meanwhile, even my simple patch [1] isn't accepted.
>
> [1] https://github.com/zeromq/libzmq/pull/240
I added a comment. You solution seems reasonable and systematically implemented.
The policy more or less requires that it be pulled, so you just need to wait.
The policy here now is more oriented to action rather than talk.
Roughly, if you think something should be done, just do it,
let people know, if they don't like it they'll say so and you can undo it,
fix it, or someone else can.
And they will :)
There's no need to fear someone will undo your changes.
On the contrary, it's a relief!
for example, as I comment on your pull request, in C++ there's another
way to do it. You chose:
void f(T unused) { (void)unused; .. }
The other way, in C++ only, is:
void f(T) { .. }
i.e. don't name the argument. So, if I feel like it I could change your patch
to the other way. I won't, the point is, there's a choice, you made it,
and someone strongly dis-agreeing might re-patch or even revert.
The advantage of that is that you don't have to feel completely responsible.
Which encourages you to actually make changes.
So basically .. if, after a while, you patch sticks and there's no complaint,
you should actually put the rule in the Style Guide, how to treat unused
arguments.
--
john skaller
skal...@users.sourceforge.net
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On Sat, Feb 11, 2012 at 11:06 AM, niXman wrote: > > Meanwhile, even my simple patch [1] isn't accepted. > > > [1] https://github.com/zeromq/libzmq/pull/240 > ... it's been a day. You need to consider resetting your expectations of open source communities. Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
2012/2/11 john skaller :
>
> On 11/02/2012, at 9:23 PM, niXman wrote:
>
>> In the example above, it is enough to add check on null pointer ( if
>> (optvallen_ && !(optval_)) {} ). This gives to the user more
>> information about the error, instead of the silent crash.
>
> Agreed.
>
>> Regarding the check of pointers of sockets, everything is simple here.
>> I'm actually very surprised that at present check is implemented in
>> this way.
>>
>> Any function fulfilling operations with sockets except zmq_socket(),
>> should check the pointer for its presence presence in ctx_t::sockets.
>> (IMHO)
>> It can seem an unnecessary overhead. But I consider that this overhead
>> will be very scanty.
>
> Well, if you believe that, you should write the rule up in the Style Guide.
> I would add the qualifier: any "public" function. I.e. a member of the
> official 0MQ C bindings.
>
> Then there's something concrete to:
>
> (a) implement
> (b) argue about
>
> I'm neutral at the moment. I can't see the harm in it.
>
> Also, as to performance: the checks using assert() can be turned
> off with -DNDEBUG already. So it may or may not make sense
> to turn other checks on or off the same way, or to re-implement the
> assert()ions so they *can't* be turned off with NDEBUG.
>
> --
> john skaller
> skal...@users.sourceforge.net
>
>
Before writing something, I want to understand the attitude of the
project Administrators to innovations, and their readiness to take
measures for improving libzmq.
Meanwhile, even my simple patch [1] isn't accepted.
[1] https://github.com/zeromq/libzmq/pull/240
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 11/02/2012, at 9:23 PM, niXman wrote:
> In the example above, it is enough to add check on null pointer ( if
> (optvallen_ && !(optval_)) {} ). This gives to the user more
> information about the error, instead of the silent crash.
Agreed.
> Regarding the check of pointers of sockets, everything is simple here.
> I'm actually very surprised that at present check is implemented in
> this way.
>
> Any function fulfilling operations with sockets except zmq_socket(),
> should check the pointer for its presence presence in ctx_t::sockets.
> (IMHO)
> It can seem an unnecessary overhead. But I consider that this overhead
> will be very scanty.
Well, if you believe that, you should write the rule up in the Style Guide.
I would add the qualifier: any "public" function. I.e. a member of the
official 0MQ C bindings.
Then there's something concrete to:
(a) implement
(b) argue about
I'm neutral at the moment. I can't see the harm in it.
Also, as to performance: the checks using assert() can be turned
off with -DNDEBUG already. So it may or may not make sense
to turn other checks on or off the same way, or to re-implement the
assert()ions so they *can't* be turned off with NDEBUG.
--
john skaller
skal...@users.sourceforge.net
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
2012/2/11 john skaller :
>
> On 11/02/2012, at 8:27 PM, niXman wrote:
>
>>
>> Continuing review of the libzmq code, I have found a few situations
>> when the program will be crushed on segmentation fault.
>
> It's impossible to avoid this in C.
>
>> I consider that programs crash on segmentation fault is an
>> inadmissible error of the ØMQ developers', but not the library user's.
>
> No. The right thing is: the library is only responsible if the
> pre-conditions of the function call are met. If the pre-conditions
> of the call are not met, all bets are off.
>
>
>> For example the call:
>>> rc = zmq_setsockopt (sub, ZMQ_SUBSCRIBE, 0, 4);
>> will end on segmentation fault on line 56 of the sub.cpp file.
>
> The user, not the library, is responsible for sending correct
> arguments to functions.
>
> I assume here, the problem is that a blob was expected, i.e. a pointer
> to a char array, length 4, but a NULL pointer got sent instead.
>
> That's the user's fault. The library might check the pointer is not
> NULL, but that's not enough. The pointer may be non-null
> and point off the end of an array: you'll get a segfault.
>
> libzmq does some really weird things to try to check sockets,
> etc, are valid things, to compensate for the stupidity of C.
> But they're only statistical helps. Sometimes it will catch
> a bad pointer. Sometimes it will segfault.
>
> It's actually not clear to me it is worth cluttering up the code
> with such sanity checks. We hope instead no one is actually
> using the C binding of 0MQ, other than bindings into saner languages.
>
> --
> john skaller
> skal...@users.sourceforge.net
>
>
>
In the example above, it is enough to add check on null pointer ( if
(optvallen_ && !(optval_)) {} ). This gives to the user more
information about the error, instead of the silent crash.
Regarding the check of pointers of sockets, everything is simple here.
I'm actually very surprised that at present check is implemented in
this way.
Any function fulfilling operations with sockets except zmq_socket(),
should check the pointer for its presence presence in ctx_t::sockets.
(IMHO)
It can seem an unnecessary overhead. But I consider that this overhead
will be very scanty.
Regards.
niXman.
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Questions about Coding Style
On 11/02/2012, at 8:27 PM, niXman wrote: > > Continuing review of the libzmq code, I have found a few situations > when the program will be crushed on segmentation fault. It's impossible to avoid this in C. > I consider that programs crash on segmentation fault is an > inadmissible error of the ØMQ developers', but not the library user's. No. The right thing is: the library is only responsible if the pre-conditions of the function call are met. If the pre-conditions of the call are not met, all bets are off. > For example the call: >> rc = zmq_setsockopt (sub, ZMQ_SUBSCRIBE, 0, 4); > will end on segmentation fault on line 56 of the sub.cpp file. The user, not the library, is responsible for sending correct arguments to functions. I assume here, the problem is that a blob was expected, i.e. a pointer to a char array, length 4, but a NULL pointer got sent instead. That's the user's fault. The library might check the pointer is not NULL, but that's not enough. The pointer may be non-null and point off the end of an array: you'll get a segfault. libzmq does some really weird things to try to check sockets, etc, are valid things, to compensate for the stupidity of C. But they're only statistical helps. Sometimes it will catch a bad pointer. Sometimes it will segfault. It's actually not clear to me it is worth cluttering up the code with such sanity checks. We hope instead no one is actually using the C binding of 0MQ, other than bindings into saner languages. -- john skaller skal...@users.sourceforge.net ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Questions about Coding Style
Hello list. Continuing review of the libzmq code, I have found a few situations when the program will be crushed on segmentation fault. On the one hand, it is possible to refer to an error because of the user. But on the other hand, Coding Style says: Assertions say that the ØMQ developers do not expect this situation to ever happen, i.e. it is an internal error that demands a fix to the ØMQ code. I consider that programs crash on segmentation fault is an inadmissible error of the ØMQ developers', but not the library user's. For example the call: >rc = zmq_setsockopt (sub, ZMQ_SUBSCRIBE, 0, 4); will end on segmentation fault on line 56 of the sub.cpp file. According to the documentation, if an error occurs, the function should return -1. I can not decide how to handle such a situation. Should I use assert or return EINVAL? Regards. niXman. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
