Re: against enforce
Jonathan M Davis wrote: On 2011-03-26 08:55, Don wrote: spir wrote: On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Denis That one is done by the CPU, as mentioned in another post. But it does illustrate something interesting: the contract programming makes you think there is a symmetry between in and out contracts, whereas in fact they have very little in common. If an out contract fails, it ALWAYS indicates a bug in the function. So it should ALWAYS be an assert. But 'in' contracts are completely different, since they indicate a problem in the calling code. (Personally I cannot see the value of 'out' contracts, except as information for verifying later 'in' contracts). Whereas I _rarely_ use in contracts. In most cases, I favor exceptions, treating my functions as API functions pretty much as long as they're public. That's not always the best approach, but it's generally what I end up using. invariants, I use fairly frequently (and in fact have run into trouble due to issues with them - e.g. http://d.puremagic.com/issues/show_bug.cgi?id=5058 ). out, on the other hand, I don't use as much. It's not terribly common that there's a good test to run on the return value of a function in my experience. Upon occasion, it's useful, but rarely. Unit tests generally solve the problem better with regards to testing output. They're still useful though - just not commonly so. So, honestly, other than invariants, I don't use D's DbC much. in contracts get used upon occasion, but I tend to favor exceptions over assertions in most cases, so I don't use in all that often, and while out is occasionally useful, unit tests generally do a better job of verifying that a function's return value is correct. I'm _very_ glad to have invariants though. - Jonathan M Davis If you only have standalone functions, at the moment there's not really any benefit to in and out contracts. You can equally well put the asserts at the start and end of the function. I totally agree with you about invariants. I've been amazed at how many bugs can be caught by even a very trivial invariant.
Re: against enforce
On 2011-03-28 00:53, Don wrote: Jonathan M Davis wrote: On 2011-03-26 08:55, Don wrote: spir wrote: On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Denis That one is done by the CPU, as mentioned in another post. But it does illustrate something interesting: the contract programming makes you think there is a symmetry between in and out contracts, whereas in fact they have very little in common. If an out contract fails, it ALWAYS indicates a bug in the function. So it should ALWAYS be an assert. But 'in' contracts are completely different, since they indicate a problem in the calling code. (Personally I cannot see the value of 'out' contracts, except as information for verifying later 'in' contracts). Whereas I _rarely_ use in contracts. In most cases, I favor exceptions, treating my functions as API functions pretty much as long as they're public. That's not always the best approach, but it's generally what I end up using. invariants, I use fairly frequently (and in fact have run into trouble due to issues with them - e.g. http://d.puremagic.com/issues/show_bug.cgi?id=5058 ). out, on the other hand, I don't use as much. It's not terribly common that there's a good test to run on the return value of a function in my experience. Upon occasion, it's useful, but rarely. Unit tests generally solve the problem better with regards to testing output. They're still useful though - just not commonly so. So, honestly, other than invariants, I don't use D's DbC much. in contracts get used upon occasion, but I tend to favor exceptions over assertions in most cases, so I don't use in all that often, and while out is occasionally useful, unit tests generally do a better job of verifying that a function's return value is correct. I'm _very_ glad to have invariants though. - Jonathan M Davis If you only have standalone functions, at the moment there's not really any benefit to in and out contracts. You can equally well put the asserts at the start and end of the function. I totally agree with you about invariants. I've been amazed at how many bugs can be caught by even a very trivial invariant. What's even more surprising is how invariants sometimes find bugs in _other_ code (e.g. http://d.puremagic.com/issues/show_bug.cgi?id=5500 ) instead of just the member functions of the type with the invariant. - Jonathan M Davis
Re: against enforce
On 3/28/2011 12:53 AM, Don wrote: If you only have standalone functions, at the moment there's not really any benefit to in and out contracts. You can equally well put the asserts at the start and end of the function. In and out contracts come into their own when you're using inheritance. The in contract is an OR of all the overridden function in's, and the out is the AND of all the overridden out's. It's pretty clunky to do this manually. They also have a possible future use as input to a static program analyzer.
Re: against enforce
Jonathan M Davis Wrote: Whereas I _rarely_ use in contracts. In most cases, I favor exceptions, treating my functions as API functions pretty much as long as they're public. That's not always the best approach, but it's generally what I end up using. So why do you need a feature, which you don't use? Also if a feature isn't used, doesn't it mean it's useless? out, on the other hand, I don't use as much. It's not terribly common that there's a good test to run on the return value of a function in my experience. Upon occasion, it's useful, but rarely. Unit tests generally solve the problem better with regards to testing output. They're still useful though - just not commonly so. If you don't prohibit contracts to check external data, you can enforce the check at design time with out contract. How would you check that caller checks received data?
Re: against enforce
On Fri, 25 Mar 2011 23:17:53 -0400, spir denis.s...@gmail.com wrote: On 03/25/2011 09:49 PM, Steven Schveighoffer wrote: On Fri, 25 Mar 2011 16:23:08 -0400, spir denis.s...@gmail.com wrote: This logic certainly looks sensible, but I cannot understand how it should work in practice. Say I'm implementing a little set of operations on decimals. Among those, some (division, square root...) will necessarily have to check their input. You can look at it in a simple way: If this branch of code will always run the same way every time (i.e. is deterministic), then using assert is desired. Why? Because the assumption is that you test that code via unit tests. Once you test it, and it works, there is no reason to test it again. For example, if I do: sqrt(1); There is never ever a need to test this in production code. sqrt(1) is always 1, and will always work. If I do: sqrt(-1); There is never ever a need to test this in production code. sqrt(-1) is always a failure, and will always fail. Unit tests should catch this code before it is released. But if I do something like: auto file = File.open(/tmp/xyz.txt); // did not look up usage for File, may be wrong I agree with all of this. But here you're the client of sqrt. What if you implement it (or for a new numeric type)? You'll need to catch param errors for /input/ to your func (not for args you /provide/ to a third party func). That was my case. The issue isn't so much that you should or should not catch errors, the issue is, can the error catching be *optimized out* at production time. The answer to that question is impossible for a compiler to answer, because figuring out where a parameter came from I believe is the halting problem, and impossible (currently) for a developer who can figure out the answer to handle. This is really a question of optimization, not error catching. If something will always be the same, and it has already been tested, there is no reason to test it again. -Steve
Re: against enforce
On Mon, 28 Mar 2011 03:53:56 -0400, Don nos...@nospam.com wrote: If you only have standalone functions, at the moment there's not really any benefit to in and out contracts. You can equally well put the asserts at the start and end of the function. Well, if you have code in your in/out contracts that is not an assert call, it will also be compiled out (not true if the in check is inside the body). -Steve
Re: against enforce
On Sat, 26 Mar 2011 08:14:19 -0400, Kagamin s...@here.lot wrote: Steven Schveighoffer Wrote: For example, if I do: sqrt(1); There is never ever a need to test this in production code. sqrt(1) is always 1, and will always work. If I do: sqrt(-1); pure function called on literals is evaluatable at compile time together with checks, so if sqrt run at runtime, this means input couldn't be validated that easily, so checks should be run too. This means that it's unnecessary to ever remove checks from sqrt. It's not that simple. for example: sqrt(uniform(1.0, 2000.0)); This is completely safe, and needs no runtime error checks. It's easier to explain with simple examples, knowing that you can extrapolate more complex ones that the compiler can't prove. -Steve
Re: against enforce
Steven Schveighoffer Wrote: This is really a question of optimization, not error catching. If something will always be the same, and it has already been tested, there is no reason to test it again. Optimization is not a question of error catching, only if it's smart enough to not interfere with error catching, but we have only blunt method of -release switch. Thus optimization becomes question of error catching.
Re: against enforce
On Mon, 28 Mar 2011 10:28:04 -0400, Kagamin s...@here.lot wrote: Steven Schveighoffer Wrote: This is really a question of optimization, not error catching. If something will always be the same, and it has already been tested, there is no reason to test it again. Optimization is not a question of error catching, only if it's smart enough to not interfere with error catching, but we have only blunt method of -release switch. Thus optimization becomes question of error catching. Yes, the instrument is too blunt, which is why we are having this discussion ;) But the question is still not *whether* to test for errors but *when*. The ideal is to avoid testing for them when they could not possibly occur (i.e. after testing during development proves there is no error). The whole point of using assert and contracts only in development code is so you can instrument all your code with comprehensive, possibly slow performing, tests while testing, and take them all out for shipping. But the mechanisms provided do not allow expression of when an error can occur (during testing or during production) *per call*, only per function. We need it per call to be correct. Made even more difficult is that we are discussing library functions, which can't possibly know whether to test the inputs they get unless the functions are private. You can take the better safe than fast approach, but it leaves no room for performance upgrades (as shown by the poor performance of phobos in comparison tests by the likes of bearophile and others). You can take the not my problem approach and leave it up to the caller to test inputs, but we don't provide an easy way for a user to test inputs in production code for when its needed at production time. My opinion is that Phobos should opt for not my problem because 90% of the time, the input comes from deterministic sources that only need be tested during development. All we need for that is a phobos compiled in non-release mode to test against. It's always possible to add more tests *outside* the phobos call for when they are needed. Then we should look into adding ways to have functions specify how to test their inputs during production in a DRY way. But of course, this path makes code that uses Phobos more prone to errors. It may be worth having all the enforce checks if we can do something like prevent all buffer overflow attacks. I'm not sure of the right answer. -Steve
Re: against enforce
On 2011-03-28 04:56, Kagamin wrote: Jonathan M Davis Wrote: Whereas I _rarely_ use in contracts. In most cases, I favor exceptions, treating my functions as API functions pretty much as long as they're public. That's not always the best approach, but it's generally what I end up using. So why do you need a feature, which you don't use? Also if a feature isn't used, doesn't it mean it's useless? 1. I do use it sometimes, just not often. 2. Just because I don't personally use it much doesn't mean that it's useless. 3. The main place that in is supposed to shine is with inheritance trees. It handles contract inheritance. out, on the other hand, I don't use as much. It's not terribly common that there's a good test to run on the return value of a function in my experience. Upon occasion, it's useful, but rarely. Unit tests generally solve the problem better with regards to testing output. They're still useful though - just not commonly so. If you don't prohibit contracts to check external data, you can enforce the check at design time with out contract. How would you check that caller checks received data? I'm afraid that I don't understand the question. out contracts are intended to verify that the result(s) of the function are correct (return value, the state of the object if it's a member variable, etc.). That's not generally very flexible in that it implies that there is a test that you can run on the result of a function to determine that the result is correct _regardless_ of the input. I don't find that to be useful very often, but sometimes it is. More often what I need is to check that given a particular input, the function gives you a particular output. And that's a job for unit tests. - Jonathan M Davis
Re: against enforce
Jonathan M Davis Wrote: So, there is a clear and distinct difference between the intended uses of assert and exceptions (and therefore enforce). They have very different roles. The question then is not what their roles are but what you need a particular function to do - e.g. treat it's input as user input or treat it as local to the program (and therefore a bug if it's wrong). I don't think it's a terminological question, it's a practical question, every piece of code should do something useful. Ok, assert checks for bugs. The question is should these checks be removed and why.
Re: against enforce
Steven Schveighoffer Wrote: For example, if I do: sqrt(1); There is never ever a need to test this in production code. sqrt(1) is always 1, and will always work. If I do: sqrt(-1); pure function called on literals is evaluatable at compile time together with checks, so if sqrt run at runtime, this means input couldn't be validated that easily, so checks should be run too. This means that it's unnecessary to ever remove checks from sqrt.
Re: against enforce
On 03/26/2011 04:31 AM, Jonathan M Davis wrote: On 2011-03-25 20:10, spir wrote: On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Don would know better than I do, but I believe that that is a CPU thing there. D isn't doing that. The CPU is. All right! Denis -- _ vita es estrany spir.wikidot.com
Re: against enforce
spir wrote: On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Denis That one is done by the CPU, as mentioned in another post. But it does illustrate something interesting: the contract programming makes you think there is a symmetry between in and out contracts, whereas in fact they have very little in common. If an out contract fails, it ALWAYS indicates a bug in the function. So it should ALWAYS be an assert. But 'in' contracts are completely different, since they indicate a problem in the calling code. (Personally I cannot see the value of 'out' contracts, except as information for verifying later 'in' contracts). When I started writing D code, I originally used 'in' contracts frequently in my mathematical code. But later, I removed most of them. Why? Because including a particular condition in a 'in' contract makes it undefined behaviour in release mode! So you need to be clear, is this parameter value a runtime error, or is it a bug in the calling code? In most mathematical code, you want it to be an error -- you don't normally want undefined behaviour for particular inputs. If the requirements for valid parameter values are complicated, you're placing a huge burden on the caller if you include asserts, making them bugs. In summary: enforce is nice for the caller, assert is nice for the callee. The decision of whom to favour can be tricky.
Re: against enforce
On 03/26/2011 04:55 PM, Don wrote: spir wrote: On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Denis That one is done by the CPU, as mentioned in another post. But it does illustrate something interesting: the contract programming makes you think there is a symmetry between in and out contracts, whereas in fact they have very little in common. If an out contract fails, it ALWAYS indicates a bug in the function. So it should ALWAYS be an assert. But 'in' contracts are completely different, since they indicate a problem in the calling code. (Personally I cannot see the value of 'out' contracts, except as information for verifying later 'in' contracts). When I started writing D code, I originally used 'in' contracts frequently in my mathematical code. But later, I removed most of them. Why? Because including a particular condition in a 'in' contract makes it undefined behaviour in release mode! So you need to be clear, is this parameter value a runtime error, or is it a bug in the calling code? In most mathematical code, you want it to be an error -- you don't normally want undefined behaviour for particular inputs. If the requirements for valid parameter values are complicated, you're placing a huge burden on the caller if you include asserts, making them bugs. In summary: enforce is nice for the caller, assert is nice for the callee. The decision of whom to favour can be tricky. Very interesting points of view, than you! Especially * making the misleading symmetry between 'in' 'out' contracts (== importance of *naming*, again!) * the relative weakness of in contracts or asserts. About 'out' contracts, I know see them as an alternate way of testing, during development or maintenance. Kinds of constant assertions (in the logical sense) checked there, which may help catching bugs, and before all do not need to be checked in unittests. Denis -- _ vita es estrany spir.wikidot.com
Re: against enforce
On 2011-03-26 08:55, Don wrote: spir wrote: On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Denis That one is done by the CPU, as mentioned in another post. But it does illustrate something interesting: the contract programming makes you think there is a symmetry between in and out contracts, whereas in fact they have very little in common. If an out contract fails, it ALWAYS indicates a bug in the function. So it should ALWAYS be an assert. But 'in' contracts are completely different, since they indicate a problem in the calling code. (Personally I cannot see the value of 'out' contracts, except as information for verifying later 'in' contracts). Whereas I _rarely_ use in contracts. In most cases, I favor exceptions, treating my functions as API functions pretty much as long as they're public. That's not always the best approach, but it's generally what I end up using. invariants, I use fairly frequently (and in fact have run into trouble due to issues with them - e.g. http://d.puremagic.com/issues/show_bug.cgi?id=5058 ). out, on the other hand, I don't use as much. It's not terribly common that there's a good test to run on the return value of a function in my experience. Upon occasion, it's useful, but rarely. Unit tests generally solve the problem better with regards to testing output. They're still useful though - just not commonly so. So, honestly, other than invariants, I don't use D's DbC much. in contracts get used upon occasion, but I tend to favor exceptions over assertions in most cases, so I don't use in all that often, and while out is occasionally useful, unit tests generally do a better job of verifying that a function's return value is correct. I'm _very_ glad to have invariants though. - Jonathan M Davis
against enforce
So, there really is no good answer. - Jonathan M Davis So why do you need to differentiate between assert and enforce if you can't choose, which of them should be used? We can't really turn off both of them, and if we really want performance and no checks, we would want to turn off both of them, so they should work in the same way, shouldn't they?
Re: against enforce
On 25/03/2011 08:04, Kagamin wrote:
So, there really is no good answer.
- Jonathan M Davis
So why do you need to differentiate between assert and enforce if you can't
choose, which of them should be used?
We can't really turn off both of them, and if we really want performance and no
checks, we would want to turn off both of them, so they should work in the same
way, shouldn't they?
I can't think of an occasion when I would want to have my uses of
enforce() stripped for a release build. Removing asserts once I am sure
that my code has no logic errors make sense, but removing the code that
validates user input and system integrity doesn't.
Performance is irrelevant if you are processing corrupted memory or
malformed input.
Before there we had enforce, I did tend to use assert for both cases as
I cobbled code together, then rewrite the validation parts as throws on
the second pass. When enforce appeared that 2 phase process got much
easier, and now I can choose the right one as I code, which increases
productivity.
Enforce does not replace assert, it compliments it, and very elegantly so.
aside
With regard to libraries, there is an overlap between what is input
validation and what is logic checking (a logic error in the code of
program a, results in bad input being sent to lib b). Here, assert is
the only choice as hitting an assert in a library (hopefully) exposes
logic errors in the program that uses the library, this is something
that you fix by reviewing your code, not by recovering at runtime and
trying again. This means that libraries (should) have 2 types of assert,
but I hesitate to suggest the creation of a synonym to help better
document the difference between them, as I find that the use of in{} and
out{} do a pretty good job of that already.
The only problems that I foresee are for libraries where the source is
not available (which I have not had an issue with yet) and libs that
wrap non-D code that does not use the same conventions. In these cases,
one will not always know if the -release flag (or equivalent) has been
used. If it has, then one may not find logic errors as one codes, if it
hasn't then one will find the errors, but will not be able to strip the
asserts when producing one's own release build.
/aside
A...
Re: against enforce
So, there really is no good answer. - Jonathan M Davis So why do you need to differentiate between assert and enforce if you can't choose, which of them should be used? We can't really turn off both of them, and if we really want performance and no checks, we would want to turn off both of them, so they should work in the same way, shouldn't they? assert and enforce serve _very_ different purposes. assert is for checking the logic of your code. It can go away, so you can't rely on it. It's simply for additional checks in your code to ensure that it's of high quality. You can't rely on it. It's also _not_ for error handling. When an assertion fails, there is a bug in your program. Exceptions - and therefore enforce - are for error handling. They are supposed to _always_ be there. Performance has nothing to do with them (other than the fact that they can obviously harm performance, which may cause you to refactor your code so that they're not necessary). Typically, exceptions - and therefore enforce - are used when validating input. That input is often completely dependent on the particular run of the program, and bad input isn't necessarily a bug in the program at all. When enforce fails, that does _not_ necessarily indicate a bug in your program, and it should _not_ be used for finding bugs. Input from the user is obviously always input, and you're going to have to check that and throw an exception on failure rather than use assertions. Input to a function which is completely local to your code is not in any API anywhere and whose input is completely controlled by your code should use assertions. At that point, if the function gets bad input, it's a bug in your code. Also, out blocks, invariants, and checks in the middle of functions typically have _nothing_ to do with input and should be assertions. If they fail it's a logic bug. The problem is when a function could be both used internally and used on user input. For instance, iota is typically given hard-coded values - iota(5, 100, 2) - but you could pass it value which was given to main - iota(5, 100, to!int(args[1]). With hard-coded values, assert is the correct solution. But with user input, enforce would be. So, which do you do? assert or enforce? In the case of iota, since it is almost always used with hard-coded values and even when it isn't, it's likely used with computed values rather than user input, so if it's wrong, it's a bug in the code rather than bad user input. The application can check (with enforce or with an if and throwing an exception or whatever) that the input is good before passing it to iota if that's what it's doing. With other functions though, it's less clear. And with every such function, a choice must be made. Should it treat its input as user input or as values local to the program? If it's user input, then it needs to use exceptions. If it's local to the program (at which point a bad value would be a bug in the program), then assert should be used. And when you're dealing with a library, it's not as clear what the best solution should be in. The fact that assertions are almost certainly going to be compiled out might make it so that you want to treat input to the library's API as user input rather than local to the program when you would choose to have those same functions be treated as local to the program if they weren't in another library (though of course, there are plenty of cases where API functions should just plain be treating input as user input regardless). So, there is a clear and distinct difference between the intended uses of assert and exceptions (and therefore enforce). They have very different roles. The question then is not what their roles are but what you need a particular function to do - e.g. treat it's input as user input or treat it as local to the program (and therefore a bug if it's wrong). - Jonathan M Davis
Re: against enforce
I can't think of an occasion when I would want to have my uses of
enforce() stripped for a release build.
Ideally enforce shouldn't have side effects, so its removal shouldn't affect
your business logic.
Removing asserts once I am sure
that my code has no logic errors make sense
Did you ever have this sense? Well, I can be sure hello world has no bugs,
but for something more complex... only if you're a programming god, but even
then your way won't suit us mortals.
Performance is irrelevant if you are processing corrupted memory or
malformed input.
I believe, sometimes it's easier to make sure that the input is correct, than
to make sure the code has no bugs.
This means that libraries (should) have 2 types of assert,
but I hesitate to suggest the creation of a synonym to help better
document the difference between them, as I find that the use of in{} and
out{} do a pretty good job of that already.
So you suggest to check iota's input in contract? They say, phobos is compiled
with -release flag, so all contracts are removed. Is it ok for D programmers?
Re: against enforce
On 03/25/2011 08:21 PM, Jonathan M Davis wrote: So, there really is no good answer. - Jonathan M Davis So why do you need to differentiate between assert and enforce if you can't choose, which of them should be used? We can't really turn off both of them, and if we really want performance and no checks, we would want to turn off both of them, so they should work in the same way, shouldn't they? assert and enforce serve _very_ different purposes. assert is for checking the logic of your code. It can go away, so you can't rely on it. It's simply for additional checks in your code to ensure that it's of high quality. You can't rely on it. It's also _not_ for error handling. When an assertion fails, there is a bug in your program. Exceptions - and therefore enforce - are for error handling. They are supposed to _always_ be there. Performance has nothing to do with them (other than the fact that they can obviously harm performance, which may cause you to refactor your code so that they're not necessary). Typically, exceptions - and therefore enforce - are used when validating input. That input is often completely dependent on the particular run of the program, and bad input isn't necessarily a bug in the program at all. When enforce fails, that does _not_ necessarily indicate a bug in your program, and it should _not_ be used for finding bugs. Input from the user is obviously always input, and you're going to have to check that and throw an exception on failure rather than use assertions. Input to a function which is completely local to your code is not in any API anywhere and whose input is completely controlled by your code should use assertions. At that point, if the function gets bad input, it's a bug in your code. Also, out blocks, invariants, and checks in the middle of functions typically have _nothing_ to do with input and should be assertions. If they fail it's a logic bug. The problem is when a function could be both used internally and used on user input. For instance, iota is typically given hard-coded values - iota(5, 100, 2) - but you could pass it value which was given to main - iota(5, 100, to!int(args[1]). With hard-coded values, assert is the correct solution. But with user input, enforce would be. So, which do you do? assert or enforce? In the case of iota, since it is almost always used with hard-coded values and even when it isn't, it's likely used with computed values rather than user input, so if it's wrong, it's a bug in the code rather than bad user input. The application can check (with enforce or with an if and throwing an exception or whatever) that the input is good before passing it to iota if that's what it's doing. With other functions though, it's less clear. And with every such function, a choice must be made. Should it treat its input as user input or as values local to the program? If it's user input, then it needs to use exceptions. If it's local to the program (at which point a bad value would be a bug in the program), then assert should be used. And when you're dealing with a library, it's not as clear what the best solution should be in. The fact that assertions are almost certainly going to be compiled out might make it so that you want to treat input to the library's API as user input rather than local to the program when you would choose to have those same functions be treated as local to the program if they weren't in another library (though of course, there are plenty of cases where API functions should just plain be treating input as user input regardless). So, there is a clear and distinct difference between the intended uses of assert and exceptions (and therefore enforce). They have very different roles. The question then is not what their roles are but what you need a particular function to do - e.g. treat it's input as user input or treat it as local to the program (and therefore a bug if it's wrong). This logic certainly looks sensible, but I cannot understand how it should work in practice. Say I'm implementing a little set of operations on decimals. Among those, some (division, square root...) will necessarily have to check their input. According to the rationale you expose, I should use assertions, since operand will nearly never be (direct) user input, instead be products of the app's logic. Then, what happens when div gets a null denominator on a release build? In practice, the issue is not that serious since I will certainly delegate to a lower-level func which itself throws. But I could also (in theory) implement it in assembly or whatnot. My point of view is if a func has preconditions on its args, then checkings simply cannot go away. Such considerations lead me to wonder whether we should not instead use exceptions/enforce everywhere for actual func arg checking, and use asserts in unittests only. Or use them also for /temporary/ additional checkings during development (similar to unittests in fact). A special case may
Re: against enforce
Kagamin: They say, phobos is compiled with -release flag, so all contracts are removed. I presume this problem will be fixed soon, probably with two versions of Phobos in the standard distribution. Otherwise D contract programming loses a significant part of its meaning and usefulness. Bye, bearophile
Re: against enforce
On Fri, 25 Mar 2011 16:23:08 -0400, spir denis.s...@gmail.com wrote:
This logic certainly looks sensible, but I cannot understand how it
should work in practice. Say I'm implementing a little set of operations
on decimals. Among those, some (division, square root...) will
necessarily have to check their input.
You can look at it in a simple way: If this branch of code will always
run the same way every time (i.e. is deterministic), then using assert is
desired. Why? Because the assumption is that you test that code via unit
tests. Once you test it, and it works, there is no reason to test it
again.
For example, if I do:
sqrt(1);
There is never ever a need to test this in production code. sqrt(1) is
always 1, and will always work.
If I do:
sqrt(-1);
There is never ever a need to test this in production code. sqrt(-1) is
always a failure, and will always fail. Unit tests should catch this code
before it is released.
But if I do something like:
auto file = File.open(/tmp/xyz.txt); // did not look up usage for File,
may be wrong
Then I always want this to be tested, even in release mode, because there
are environments beyond the program's control that will cause an error.
Similar to this, user input needs to be validated even in production code.
So if you had:
int main(string[] args)
{
sqrt(to!int(args[0]));
}
this means args[0] should be checked that it is a valid positive integer.
The problem with this whole utopian idea is, the code that is responsible
for knowing how to check its input is the function. The code that knows
whether the input needs to be checked during release is the caller. So
there is no way to convey to sqrt hey, this input I'm sending in is
externally-generated, check it during production too, ok?. If we had
options for that, the API quickly becomes unwieldly.
Of course, knowing that sqrt takes only positive numbers, we could do this
check outside the sqrt function, but far more complex functions may not be
so easy to validate input for. And why should the caller be responsible
for validating the input?
What I feel like we need is a compiler feature like:
validate(sqrt(to!int(args[0]));
which essentially calls sqrt with asserts and contracts *enabled*, even at
production time. This way you could choose while writing code how
essential a particular call is to check the input, but re-use the
already-in-place contracts for that function.
This might be a feature that's totally not worth the effort. But I can't
see a good way to solve this without having some sort of mechanism. It
also shouldn't throw an AssertError, since that would kill the program
instantly.
-Steve
Re: against enforce
On 03/25/2011 08:21 PM, Jonathan M Davis wrote: So, there really is no good answer. - Jonathan M Davis So why do you need to differentiate between assert and enforce if you can't choose, which of them should be used? We can't really turn off both of them, and if we really want performance and no checks, we would want to turn off both of them, so they should work in the same way, shouldn't they? assert and enforce serve _very_ different purposes. assert is for checking the logic of your code. It can go away, so you can't rely on it. It's simply for additional checks in your code to ensure that it's of high quality. You can't rely on it. It's also _not_ for error handling. When an assertion fails, there is a bug in your program. Exceptions - and therefore enforce - are for error handling. They are supposed to _always_ be there. Performance has nothing to do with them (other than the fact that they can obviously harm performance, which may cause you to refactor your code so that they're not necessary). Typically, exceptions - and therefore enforce - are used when validating input. That input is often completely dependent on the particular run of the program, and bad input isn't necessarily a bug in the program at all. When enforce fails, that does _not_ necessarily indicate a bug in your program, and it should _not_ be used for finding bugs. Input from the user is obviously always input, and you're going to have to check that and throw an exception on failure rather than use assertions. Input to a function which is completely local to your code is not in any API anywhere and whose input is completely controlled by your code should use assertions. At that point, if the function gets bad input, it's a bug in your code. Also, out blocks, invariants, and checks in the middle of functions typically have _nothing_ to do with input and should be assertions. If they fail it's a logic bug. The problem is when a function could be both used internally and used on user input. For instance, iota is typically given hard-coded values - iota(5, 100, 2) - but you could pass it value which was given to main - iota(5, 100, to!int(args[1]). With hard-coded values, assert is the correct solution. But with user input, enforce would be. So, which do you do? assert or enforce? In the case of iota, since it is almost always used with hard-coded values and even when it isn't, it's likely used with computed values rather than user input, so if it's wrong, it's a bug in the code rather than bad user input. The application can check (with enforce or with an if and throwing an exception or whatever) that the input is good before passing it to iota if that's what it's doing. With other functions though, it's less clear. And with every such function, a choice must be made. Should it treat its input as user input or as values local to the program? If it's user input, then it needs to use exceptions. If it's local to the program (at which point a bad value would be a bug in the program), then assert should be used. And when you're dealing with a library, it's not as clear what the best solution should be in. The fact that assertions are almost certainly going to be compiled out might make it so that you want to treat input to the library's API as user input rather than local to the program when you would choose to have those same functions be treated as local to the program if they weren't in another library (though of course, there are plenty of cases where API functions should just plain be treating input as user input regardless). So, there is a clear and distinct difference between the intended uses of assert and exceptions (and therefore enforce). They have very different roles. The question then is not what their roles are but what you need a particular function to do - e.g. treat it's input as user input or treat it as local to the program (and therefore a bug if it's wrong). This logic certainly looks sensible, but I cannot understand how it should work in practice. Say I'm implementing a little set of operations on decimals. Among those, some (division, square root...) will necessarily have to check their input. According to the rationale you expose, I should use assertions, since operand will nearly never be (direct) user input, instead be products of the app's logic. Then, what happens when div gets a null denominator on a release build? In practice, the issue is not that serious since I will certainly delegate to a lower-level func which itself throws. But I could also (in theory) implement it in assembly or whatnot. My point of view is if a func has preconditions on its args, then checkings simply cannot go away. Such considerations lead me to wonder whether we should not instead use exceptions/enforce everywhere for actual func arg checking, and use asserts in
Re: against enforce
On 25/03/2011 20:23, spir wrote: This logic certainly looks sensible, but I cannot understand how it should work in practice. Say I'm implementing a little set of operations on decimals. Among those, some (division, square root...) will necessarily have to check their input. According to the rationale you expose, I should use assertions, since operand will nearly never be (direct) user input, instead be products of the app's logic. Then, what happens when div gets a null denominator on a release build? In practice, the issue is not that serious since I will certainly delegate to a lower-level func which itself throws. But I could also (in theory) implement it in assembly or whatnot. My point of view is if a func has preconditions on its args, then checkings simply cannot go away. Such considerations lead me to wonder whether we should not instead use exceptions/enforce everywhere for actual func arg checking, and use asserts in unittests only. Or use them also for /temporary/ additional checkings during development (similar to unittests in fact). A special case may be about checkings that control logical values or ranges which do not prevent the func to run. Say, a.length should logically be in 1..9 -- but the func can run fine anyway. Denis Since I started programming in D, unittests have been among the first things I write, and I am just about getting the hang of using DMD's coverage features to make sure that my tests cover every branch of my code. Recently I have been trying to write unittests without using any additional asserts, making unittests into examples of use, designed to hit every corner case while telling a story. I don't find it quite as easy as just writing asserts, but it is a half step toward writing decent documentation, (something I often fail at ) and it makes the code to be tested easier to write. With the iota case, I want it to use assert, not enforce. I write my test so that when it runs it hits the corner cases and if there is a problem with the arguments sent to iota then execution stops in the library code. In this scenario, I know there is something I need to fix in my code (or possibly in Phobos ^^ ). If iota instead threw an exception, I would then have to put in a try/catch or a scope() and try to recover (to do otherwise would be to duplicate code already in the library), but that doesn't fix the problem, it just masks it, so my program has gotten worse, not better! That a compiled lib (including the Phobos lib distributed with DMD) is assert-less for performance reasons, is a quality of implementation issue, not a fault of enforce, or something it was ever intended to fix. A...
Re: against enforce
On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Denis -- _ vita es estrany spir.wikidot.com
Re: against enforce
On 03/25/2011 09:49 PM, Steven Schveighoffer wrote:
On Fri, 25 Mar 2011 16:23:08 -0400, spir denis.s...@gmail.com wrote:
This logic certainly looks sensible, but I cannot understand how it should
work in practice. Say I'm implementing a little set of operations on
decimals. Among those, some (division, square root...) will necessarily have
to check their input.
You can look at it in a simple way: If this branch of code will always run the
same way every time (i.e. is deterministic), then using assert is desired. Why?
Because the assumption is that you test that code via unit tests. Once you test
it, and it works, there is no reason to test it again.
For example, if I do:
sqrt(1);
There is never ever a need to test this in production code. sqrt(1) is always
1, and will always work.
If I do:
sqrt(-1);
There is never ever a need to test this in production code. sqrt(-1) is always
a failure, and will always fail. Unit tests should catch this code before it is
released.
But if I do something like:
auto file = File.open(/tmp/xyz.txt); // did not look up usage for File, may
be wrong
I agree with all of this. But here you're the client of sqrt. What if you
implement it (or for a new numeric type)? You'll need to catch param errors for
/input/ to your func (not for args you /provide/ to a third party func). That
was my case.
Then I always want this to be tested, even in release mode, because there are
environments beyond the program's control that will cause an error. Similar to
this, user input needs to be validated even in production code.
So if you had:
int main(string[] args)
{
sqrt(to!int(args[0]));
}
this means args[0] should be checked that it is a valid positive integer.
Agreed. Anyway, validating input, wherever it comes from, as soon as possible,
is certainly better practice. (obviously makes debugging easier)
The problem with this whole utopian idea is, the code that is responsible for
knowing how to check its input is the function. The code that knows whether the
input needs to be checked during release is the caller. So there is no way to
convey to sqrt hey, this input I'm sending in is externally-generated, check
it during production too, ok?. If we had options for that, the API quickly
becomes unwieldly.
Of course, knowing that sqrt takes only positive numbers, we could do this
check outside the sqrt function, but far more complex functions may not be so
easy to validate input for. And why should the caller be responsible for
validating the input?
What I feel like we need is a compiler feature like:
validate(sqrt(to!int(args[0]));
which essentially calls sqrt with asserts and contracts *enabled*, even at
production time. This way you could choose while writing code how essential a
particular call is to check the input, but re-use the already-in-place
contracts for that function.
This might be a feature that's totally not worth the effort. But I can't see a
good way to solve this without having some sort of mechanism. It also shouldn't
throw an AssertError, since that would kill the program instantly.
Same for me. I'd like to find a language that implements this and see how the
feature stands in front of real situations, and on the long run (remember
java's checked expections ;-)
Denis
--
_
vita es estrany
spir.wikidot.com
Re: against enforce
On 2011-03-25 20:10, spir wrote: On 03/25/2011 11:20 PM, Jonathan M Davis wrote: In the case of something like dividing by 0 or other math functions that could be given bad values, the typical solution is to either use an assertion (or check nothing) and then let the caller worry about it. It would be extremely wasteful to have to constantly check whether the arguments to typical math functions are valid. They almost always are, and those types of functions needto be really efficient. But catching wrong arguments to math functions at *runtime* is precisely what D itself does (as well as all languages I know): auto a = 1, b = 0; auto c = a/b; == Floating point exception There is no way out, or do I miss a point? Don would know better than I do, but I believe that that is a CPU thing there. D isn't doing that. The CPU is. - Jonathan M Davis
Re: Against enforce()
So how do you solve the problem? - This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. You can't validate all user input, so external data ends up spead across your entire application. So I don't understand obsession with -release switch, because contracts most of the time do validate user input. If we think about -release switch as a HP-hack for exotic code, there will be no ideological difference between assert and enforce. As has been point out, the problem is in cases where it's not clear whether you should treat input as user input (and therefore needs to _always_ be checked and have exceptions thrown on error) or whether you should treat input as being from your program and guaranteed to be valid (at which point you use assert to check that that guarantee actually holds). --
Re: Against enforce()
So how do you solve the problem? - This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. You can't validate all user input, so external data ends up spead across your entire application. So I don't understand obsession with -release switch, because contracts most of the time do validate user input. If we think about -release switch as a HP-hack for exotic code, there will be no ideological difference between assert and enforce. As has been point out, the problem is in cases where it's not clear whether you should treat input as user input (and therefore needs to _always_ be checked and have exceptions thrown on error) or whether you should treat input as being from your program and guaranteed to be valid (at which point you use assert to check that that guarantee actually holds). -- It's a case by case thing. In some cases, you go with assertions and let the code choke horribly (or worse, silently sort of work but not quite right) if it's used in a case where it should have been an exception. In others, you use exceptions and just let the efficiency be degraded. It depends on the situation and what you're trying to do. In many cases, you'd go with an assertion and make it clear that the caller needs to check if they want the function to actually work correctly. Then it's up to the caller to check or not. There is no good answer for what to _always_ do in this sort of situation, because the costs can vary considerably from situation to situation. If the function is very cheap, then having additional checks in release mode could be devastating, and you just can't afford to be throwing exceptions from it. On the other hand, if it's very expensive, then having the additional checks wouldn't matter at all. The big question is what the general policy should be in Phobos functions. Should the default choice be to use an assertion, which will usually then do no checks at all, because assertions are almost always compiled out in Phobos (unless people compile it themselves), or should enforce be used and then have the additional cost in there all of the time. I would guess that iota is most frequently used with known values at compile time - iota(1, 512) - in which case assert makes perfect sense. In others, the value used could be based on lots of calculations somewhere and maybe even depends on user input - iota(a, b). The best choice would depend on what we expect the typical use case to be and how high the cost is to pick the other choice. The other possibility is to specifically have two versions of a function: one which uses an assertion (which may or may not be enabled) and therefore essentially does no checking, thereby requiring that the programmer ensure that the arguments are correct or the function could do funny things, and one which uses an exception. Then the programmer could choose whether they want the checks to occur or not (e.g. iota could assert or do no checks at all and iotoE could throw an exception). However, that doesn't scale very well if you try and do that with every function that has this problem. So, there really is no good answer. - Jonathan M Davis
Re: Against enforce()
Steven Schveighoffer wrote:
On Mon, 21 Mar 2011 10:08:56 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
Will there not be an expectation that a pure function will not
read/write shared data that will be broken (i.e. why did the compiler
allow this, I thought I was safe from this!)?
If you pass the address of global or shared data into a weakly pure
function, it won't be pure. If you have a function with a strongly
pure signature, global data can never be passed to it.
Just on this one point, TLS or __gshared references are
indistinguishable from normal references, but shared references are not
the same, the type system lets you know it's shared. I thought this was
the main reason why we were allowed to create weak-pure functions,
because d has this distinction. Are you allowed to declare a function
like this?
int foo(shared int *x) pure {return *x;}
No.
I was under the impression that a pure function couldn't access shared
data -- period. Is this not true?
Not necessarily. It only guarantees that it won't access shared data
unless you provided it.
Clearly, this theoretically
weak-pure function could not be called from a strong-pure function, so
the pure decoration is useless.
But this is very similar to a delegate that does the same thing.
Why does one make sense and the other not? In other words, if I have a
function like this:
int foo(int delegate() x) pure {...}
is this *ever* callable from a strong-pure function? Or does the
delegate have to be declared pure? It seems to me that either:
1) it's not ever callable from a strong-pure function, making the pure
decoration useless or
It will only be callable from a strong-pure function if (a) the delegate
is marked as pure; or (b) it's a delegate literal (so its purity can be
checked).
Lazy is an instance of (b).
2) it's callable from a strong pure function, but then the compiler
needs to generate two copies of the function, one with a pure delegate,
one without.
If the delegate isn't pure, it cannot be called at all from a strongly
pure function.
Pure functions (even weakly pure) cannot call non-pure delegates in any
way -- except in the aforementioned delegate literal case.
On item 2, the reason I feel this way is in the case where foo wants to
pass the delegate to another pure function, it might optimize that call
differently if the delegate is known to be pure. Or maybe we
don't/can't care...
Thanks for having patience, I'm asking all these questions because I
want to make sure we do not put in rules that are wrong but hard to
remove later when everything uses them -- even though I don't understand
everything going on here ;)
-Steve
Re: Against enforce()
On Fri, 18 Mar 2011 20:06:16 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 14:35:27 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 04:34:54 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer Wrote:
As long as the delegate does not access shared/global data, it
should be able to be pure. Even delegates which modify TLS data
should be able to be pure (weak-pure, but still pure).
TLS variables are global and must not be accessed from any function
marked as pure. With regard to purity, there isn't any difference
between shared and TLS variables.
However, it's still not shared.
This, for example, is a weak pure function:
void foo(int *n) pure { *n = 5;}
Because TLS variables are not shared, you should be able to do this:
int x;
void bar()
{
foo(x);
}
Yes, that compiles fine. But bar() is not pure.
But you are right, there is a huge difference between a local
reference to TLS data and directly accessing TLS data -- the latter
can be obscured from the compiler, resulting in the compiler thinking
the function can be strong pure.
So I don't know exactly how to mitigate this, but in my mind, it
feels like this should work:
int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
int x;
void bar()
{
foo(x == 4, x = 5);
}
It seems not too different from the above example where you pass the
address of x. But obviously the x = 5 delegate cannot be pure (it
modifies TLS data).
We may have no recourse to get this to work. It may be a lost
cause, and you just can't have lazy variables for pure functions.
It's not a lost cause, it's a two-liner!
mtype.c line 5045:
if (fparam-storageClass STClazy)
{
-error(0, cannot have lazy parameters to a pure
function);
+tf-purity = PUREweak;
+break;
}
This is a bit conservative: it would be possible to allow lazy
parameters to be marked as pure, which would allow them to be strongly
pure. But that would probably not be worth the extra complexity.
I'm not sure this works. Aren't you allowed to pass in a delegate to
a lazy parameter?
Yes.
For example:
shared int x;
int foo()
{
return x;
}
int bar(lazy int n) pure
{
return n;
}
void baz()
{
bar(foo);
}
or alternatively:
void baz()
{
bar(x);
}
This compiles just fine. (Well, you need to use bar(foo) not bar(foo)).
But if you try to mark baz() as pure, here's what you get:
test0.d(135): Error: pure function 'baz' cannot call impure function
'foo'
But does this make sense? A pure function (bar) is reading a shared
integer via foo, I thought that was a big no-no?
or for the second case:
test0.d(136): Error: pure function 'baz' cannot access mutable static
data 'x'
bar is just weakly pure.
But I wasn't saying baz is pure, I was saying bar is pure (probably should
be more diverse in the names). But I'm concerned about a pure function
being able to indirectly read/write shared data. Does this make sense? I
guess a weak-pure function acts like a normal function when called from a
normal-function. Is that why it's ok? Will there not be an expectation
that a pure function will not read/write shared data that will be broken
(i.e. why did the compiler allow this, I thought I was safe from this!)?
So is the rule that if you pass a non-pure delegate into a pure function
it's automatically weak-pure? If so, does this not mean we need two
versions of pure functions that take delegates, one that takes a pure
delegate, and one that takes a non-pure one? Otherwise, how do you know
what's strong and what's weak? For example, a weak pure function is
strong when called from a strong-pure function, so you could say if the
calling function is pure, the call is strong-pure. But wouldn't you need
separate generated code for the two cases?
I guess you can see from the number of question marks, I'm not sure about
this at all, either way :) If you think it will work, then I trust your
judgment.
-Steve
Re: Against enforce()
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 20:06:16 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 14:35:27 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 04:34:54 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer Wrote:
As long as the delegate does not access shared/global data, it
should be able to be pure. Even delegates which modify TLS
data should be able to be pure (weak-pure, but still pure).
TLS variables are global and must not be accessed from any
function marked as pure. With regard to purity, there isn't any
difference between shared and TLS variables.
However, it's still not shared.
This, for example, is a weak pure function:
void foo(int *n) pure { *n = 5;}
Because TLS variables are not shared, you should be able to do this:
int x;
void bar()
{
foo(x);
}
Yes, that compiles fine. But bar() is not pure.
But you are right, there is a huge difference between a local
reference to TLS data and directly accessing TLS data -- the latter
can be obscured from the compiler, resulting in the compiler
thinking the function can be strong pure.
So I don't know exactly how to mitigate this, but in my mind, it
feels like this should work:
int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
int x;
void bar()
{
foo(x == 4, x = 5);
}
It seems not too different from the above example where you pass
the address of x. But obviously the x = 5 delegate cannot be pure
(it modifies TLS data).
We may have no recourse to get this to work. It may be a lost
cause, and you just can't have lazy variables for pure functions.
It's not a lost cause, it's a two-liner!
mtype.c line 5045:
if (fparam-storageClass STClazy)
{
-error(0, cannot have lazy parameters to a pure
function);
+tf-purity = PUREweak;
+break;
}
This is a bit conservative: it would be possible to allow lazy
parameters to be marked as pure, which would allow them to be
strongly pure. But that would probably not be worth the extra
complexity.
I'm not sure this works. Aren't you allowed to pass in a delegate
to a lazy parameter?
Yes.
For example:
shared int x;
int foo()
{
return x;
}
int bar(lazy int n) pure
{
return n;
}
void baz()
{
bar(foo);
}
or alternatively:
void baz()
{
bar(x);
}
This compiles just fine. (Well, you need to use bar(foo) not bar(foo)).
But if you try to mark baz() as pure, here's what you get:
test0.d(135): Error: pure function 'baz' cannot call impure function
'foo'
But does this make sense? A pure function (bar) is reading a shared
integer via foo, I thought that was a big no-no?
or for the second case:
test0.d(136): Error: pure function 'baz' cannot access mutable static
data 'x'
bar is just weakly pure.
But I wasn't saying baz is pure, I was saying bar is pure (probably
should be more diverse in the names). But I'm concerned about a pure
function being able to indirectly read/write shared data. Does this
make sense? I guess a weak-pure function acts like a normal function
when called from a normal-function. Is that why it's ok?
Yes.
Will there
not be an expectation that a pure function will not read/write shared
data that will be broken (i.e. why did the compiler allow this, I
thought I was safe from this!)?
If you pass the address of global or shared data into a weakly pure
function, it won't be pure. If you have a function with a strongly pure
signature, global data can never be passed to it.
So is the rule that if you pass a non-pure delegate into a pure function
it's automatically weak-pure?
Yes.
If so, does this not mean we need two
versions of pure functions that take delegates, one that takes a pure
delegate, and one that takes a non-pure one? Otherwise, how do you know
what's strong and what's weak? For example, a weak pure function is
strong when called from a strong-pure function, so you could say if the
calling function is pure, the call is strong-pure. But wouldn't you
need separate generated code for the two cases?
Generally not. Basically, all weak-pure means, is that it's OK for a
strongly pure function to call it. And that's pretty much all it means.
If it's called from outside a strongly pure function, it's just a normal
impure function.
Now, there is some potential for a call to a weakly pure function to be
considered as strongly pure, even when called from a normal function, if
all the parameters are pure. But that's a secondary effect (and the
compiler currently never does it): the real value of weakly pure is that
it allows a dramatic increase in the number of strongly pure functions.
I guess you can see from the number of question marks, I'm not sure
about this at all, either way :) If you think it will work, then I
trust your judgment.
-Steve
Re: Against enforce()
On Mon, 21 Mar 2011 10:08:56 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
Will there not be an expectation that a pure function will not
read/write shared data that will be broken (i.e. why did the compiler
allow this, I thought I was safe from this!)?
If you pass the address of global or shared data into a weakly pure
function, it won't be pure. If you have a function with a strongly pure
signature, global data can never be passed to it.
Just on this one point, TLS or __gshared references are indistinguishable
from normal references, but shared references are not the same, the type
system lets you know it's shared. I thought this was the main reason why
we were allowed to create weak-pure functions, because d has this
distinction. Are you allowed to declare a function like this?
int foo(shared int *x) pure {return *x;}
I was under the impression that a pure function couldn't access shared
data -- period. Is this not true? Clearly, this theoretically weak-pure
function could not be called from a strong-pure function, so the pure
decoration is useless.
But this is very similar to a delegate that does the same thing.
Why does one make sense and the other not? In other words, if I have a
function like this:
int foo(int delegate() x) pure {...}
is this *ever* callable from a strong-pure function? Or does the delegate
have to be declared pure? It seems to me that either:
1) it's not ever callable from a strong-pure function, making the pure
decoration useless or
2) it's callable from a strong pure function, but then the compiler needs
to generate two copies of the function, one with a pure delegate, one
without.
On item 2, the reason I feel this way is in the case where foo wants to
pass the delegate to another pure function, it might optimize that call
differently if the delegate is known to be pure. Or maybe we don't/can't
care...
Thanks for having patience, I'm asking all these questions because I want
to make sure we do not put in rules that are wrong but hard to remove
later when everything uses them -- even though I don't understand
everything going on here ;)
-Steve
Re: Against enforce()
On Mon, 21 Mar 2011 15:35:58 +0100, Steven Schveighoffer
schvei...@yahoo.com wrote:
Why does one make sense and the other not? In other words, if I have a
function like this:
int foo(int delegate() x) pure {...}
is this *ever* callable from a strong-pure function? Or does the
delegate have to be declared pure? It seems to me that either:
1) it's not ever callable from a strong-pure function, making the pure
decoration useless or
2) it's callable from a strong pure function, but then the compiler
needs to generate two copies of the function, one with a pure delegate,
one without.
On item 2, the reason I feel this way is in the case where foo wants to
pass the delegate to another pure function, it might optimize that call
differently if the delegate is known to be pure. Or maybe we
don't/can't care...
That is a good point. However, pure delegates should (but might not
currently) be implicitly castable to impure. Hence, one version, taking
an impure delegate, should be enough.
Moreover, I believe a pure delegate could only ever be weakly-pure, hence
precluding the use of aggressive optimizations.
If my first assumption is correct, and the latter is not, the problem
should only occur if you have two versions of a function, one taking a
pure delegate, the other taking an impure one.
--
Simen
Re: Against enforce()
Steven Schveighoffer wrote: On Thu, 17 Mar 2011 13:30:21 -0400, Simen kjaeraas simen.kja...@gmail.com wrote: On Thu, 17 Mar 2011 18:17:08 +0100, Steven Schveighoffer schvei...@yahoo.com wrote: On Thu, 17 Mar 2011 12:09:50 -0400, bearophile bearophileh...@lycos.com wrote: Steven Schveighoffer: The only problem I see with it is the inline-killing. Please don't ignore the purity-killing :-) I think this is not as much an easy fix. By changing one line in enforce, every instance becomes inlinable. By making enforce also pure, it doesn't automatically make all users of enforce pure. I thought that lazy enforce cannot be pure, but I realize now that it can, as long as the delegate is pure. However, I think the compiler won't cooperate with that. Not currently, at least. This made me wonder. A delegate created inside a pure function would have to be pure while in the scope of that function, right? Seems to me that should be possible to implement. As long as the delegate does not access shared/global data, it should be able to be pure. Even delegates which modify TLS data should be able to be pure (weak-pure, but still pure). This should be easy to enforce when the delegate is created automatically from an expression using a lazy call. However, we need some implicit casting rules for pure delegates into non-pure ones. -Steve Fortunately, we don't. Delegates created via lazy are not the same as external delegates; they're an even simpler case of delegate literals. This means that at the moment that a lazy function is called, all of the code of the delegate is available. Furthermore, the caller function must access every expression inside the lazy delegate; if the caller is marked as pure, every expression inside the lazy delegate must also be pure. So the existing checks work just fine. Patch in bug 5750.
Re: Against enforce()
Steven Schveighoffer Wrote: As long as the delegate does not access shared/global data, it should be able to be pure. Even delegates which modify TLS data should be able to be pure (weak-pure, but still pure). Pure functions calling weakly pure functions are also weakly pure and so on. This effectively leaves you without purity.
Re: Against enforce()
Walter Bright Wrote:
1. Asserts and contracts are for detecting program BUGS. They are not for
validating user input, checking for disk full, file not found errors, etc.
2. Enforce is for validating user input, checking for disk full, file not
found
errors, etc. Enforce is NOT for use in contracts or checking for program bugs.
Any use of enforce in Phobos that is checking for program bugs is itself a
bug
and should be entered into bugzilla for fixing.
So this is a bug? This is a contract, not a validation of user input.
struct Iota(N, S) if ((isIntegral!N || isPointer!N) isIntegral!S)
{
private N current, pastLast;
private S step;
this(N current, N pastLast, S step)
{
enforce((current = pastLast step 0) ||
(current = pastLast step 0));
this.current = current;
this.step = step;
Re: Against enforce()
Kagamin wrote: Steven Schveighoffer Wrote: As long as the delegate does not access shared/global data, it should be able to be pure. Even delegates which modify TLS data should be able to be pure (weak-pure, but still pure). TLS variables are global and must not be accessed from any function marked as pure. With regard to purity, there isn't any difference between shared and TLS variables. Pure functions calling weakly pure functions are also weakly pure and so on. This effectively leaves you without purity. I presume you mean Pure functions calling weakly pure functions *would also be* weakly pure and so on. ?
Re: Against enforce()
Don Wrote: Pure functions calling weakly pure functions are also weakly pure and so on. This effectively leaves you without purity. I presume you mean Pure functions calling weakly pure functions *would also be* weakly pure and so on. ? What's the difference?
Re: Against enforce()
On Fri, 18 Mar 2011 03:50:17 -0400, Kagamin s...@here.lot wrote: Steven Schveighoffer Wrote: As long as the delegate does not access shared/global data, it should be able to be pure. Even delegates which modify TLS data should be able to be pure (weak-pure, but still pure). Pure functions calling weakly pure functions are also weakly pure and so on. This effectively leaves you without purity. No. Strong-pure functions can call weak-pure functions and still can be strong-pure. That's the huge benefit of weak-pure functions -- you can modularize pure functions without having to change everything to immutable. -Steve
Re: Against enforce()
On Fri, 18 Mar 2011 04:34:54 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer Wrote:
As long as the delegate does not access shared/global data, it should
be able to be pure. Even delegates which modify TLS data should be
able to be pure (weak-pure, but still pure).
TLS variables are global and must not be accessed from any function
marked as pure. With regard to purity, there isn't any difference
between shared and TLS variables.
However, it's still not shared.
This, for example, is a weak pure function:
void foo(int *n) pure { *n = 5;}
Because TLS variables are not shared, you should be able to do this:
int x;
void bar()
{
foo(x);
}
But you are right, there is a huge difference between a local reference to
TLS data and directly accessing TLS data -- the latter can be obscured
from the compiler, resulting in the compiler thinking the function can be
strong pure.
So I don't know exactly how to mitigate this, but in my mind, it feels
like this should work:
int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
int x;
void bar()
{
foo(x == 4, x = 5);
}
It seems not too different from the above example where you pass the
address of x. But obviously the x = 5 delegate cannot be pure (it
modifies TLS data).
We may have no recourse to get this to work. It may be a lost cause, and
you just can't have lazy variables for pure functions.
Cue the request for macros where you can just rewrite the syntax vs.
having to use lazy in the first place...
-Steve
Re: Against enforce()
On Fri, 18 Mar 2011 04:14:12 -0400, Kagamin s...@here.lot wrote:
Walter Bright Wrote:
1. Asserts and contracts are for detecting program BUGS. They are not
for
validating user input, checking for disk full, file not found errors,
etc.
2. Enforce is for validating user input, checking for disk full, file
not found
errors, etc. Enforce is NOT for use in contracts or checking for
program bugs.
Any use of enforce in Phobos that is checking for program bugs is
itself a bug
and should be entered into bugzilla for fixing.
So this is a bug? This is a contract, not a validation of user input.
struct Iota(N, S) if ((isIntegral!N || isPointer!N) isIntegral!S)
{
private N current, pastLast;
private S step;
this(N current, N pastLast, S step)
{
enforce((current = pastLast step 0) ||
(current = pastLast step 0));
this.current = current;
this.step = step;
This is a good example of why it's difficult to decide what user input
is. One could consider that the 'user' in this case is the developer
using the library, but I don't think that's the right choice.
I'd say it's a bug, this is clearly a contract, since the data being
passed into the ctor can easily not be user input (i.e. it's most likely
two literals that will never depend on a user). If it is user input, the
caller of the ctor should enforce the user input before passing it to iota.
-Steve
Re: Against enforce()
On 03/18/2011 01:37 PM, Steven Schveighoffer wrote: This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. This is indeed a difficult topic. I'm a bit bluffed when reading people confidently asserting apparently clear positions about the use of enforce vs assert vs contracts and such, or whether such checks should or not stay or not in various distribution builds (mainly -release). I can see at least 5 cases, and am far to be sure what the proper tool is in every case, and in which builds it should stay. In each case, there is potential wrong input; but note the variety of cases does seems orthogonal (lol) to what kind of harm it may cause: * colleague: my code is called by code from the same app (same dev team); typically, wrong input logically cannot happen * friend: my code is called by code designed to cooperate with it; there is a kind of moral contract In both cases, wrong input reveals a bug; but in the first case, it's my own (team's) bug. I guess, but am not sure, these cases are good candidates for asserts (or contracts?), excluded from release build. * lib call: my code is a typical lib; thus, I have zero control on caller. I would let the check in release mode, thus use enforce. Or, use assert if it remains when the *caller* is compiled in debug mode. There is something unclear here, I guess. Maybe there are two sub-cases: ~ the caller logically should be able to prove its args correct ~ or not In the first case, checks should dispappear in release mode. But there is always a risk. So, what to choose if my func really requires correct input? By the way, I don't understand the diff between enforce and alwaysAssert; I thought enforce was precisely an alwaysAssert. * user input: cope with it. It's the only clear case for me. Denis -- _ vita es estrany spir.wikidot.com
Re: Against enforce()
On Fri, 18 Mar 2011 11:35:22 -0400, spir denis.s...@gmail.com wrote: On 03/18/2011 01:37 PM, Steven Schveighoffer wrote: This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. This is indeed a difficult topic. I'm a bit bluffed when reading people confidently asserting apparently clear positions about the use of enforce vs assert vs contracts and such, or whether such checks should or not stay or not in various distribution builds (mainly -release). I can see at least 5 cases, and am far to be sure what the proper tool is in every case, and in which builds it should stay. In each case, there is potential wrong input; but note the variety of cases does seems orthogonal (lol) to what kind of harm it may cause: * colleague: my code is called by code from the same app (same dev team); typically, wrong input logically cannot happen * friend: my code is called by code designed to cooperate with it; there is a kind of moral contract In both cases, wrong input reveals a bug; but in the first case, it's my own (team's) bug. I guess, but am not sure, these cases are good candidates for asserts (or contracts?), excluded from release build. * lib call: my code is a typical lib; thus, I have zero control on caller. I would let the check in release mode, thus use enforce. Or, use assert if it remains when the *caller* is compiled in debug mode. There is something unclear here, I guess. Maybe there are two sub-cases: ~ the caller logically should be able to prove its args correct ~ or not See, this is where I feel we have issues. The clear problem with *always* checking is the iota example. One may use iota like this: foreach(i; iota(0, 5)) Why should checks in iota remain for iota to prove that 0 is less than 5? It always will be less than 5, and the check is not necessary. checks should only be in place during release when the input to the function cannot be proven at compile time. When it can be proven, then the checks should go away. The problem I see is it's iota's responsibility to do those checks, but it has no idea where the data comes from. What I would suggest is to check at the point the argument data is created, not at the point where it's used. So for instance, if you get the parameters for iota from an input file, then you need to check those arguments before passing to iota. This is a difficult problem to solve, because one party knows whether the arguments need to be checked, and the other party knows how to check the arguments. I don't know if there is a clean way to do this. My thoughts are that phobos should only use enforce where it can prove the arguments are runtime-generated, and rely on asserts otherwise. The obvious pitfall is that one could pass runtime-generated data to a phobos function which uses asserts, and the program could crash on an otherwise recoverable error because the user of phobos did not validate the input first. I think the risk here is less important than the reduction in performance that occurs when enforce is used instead. By the way, I don't understand the diff between enforce and alwaysAssert; I thought enforce was precisely an alwaysAssert. Assert throws an unrecoverable error, and its indication is that there is a problem in the code. An enforce throws a recoverable exception, and indicates there is a problem in the runtime input. -Steve
Re: Against enforce()
Steven Schveighoffer Wrote: This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. You can't validate all user input, so external data ends up spead across your entire application. So I don't understand obsession with -release switch, because contracts most of the time do validate user input. If we think about -release switch as a HP-hack for exotic code, there will be no ideological difference between assert and enforce.
Re: Against enforce()
On 3/18/11 11:07 AM, Steven Schveighoffer wrote: On Fri, 18 Mar 2011 11:35:22 -0400, spir denis.s...@gmail.com wrote: On 03/18/2011 01:37 PM, Steven Schveighoffer wrote: This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. This is indeed a difficult topic. I'm a bit bluffed when reading people confidently asserting apparently clear positions about the use of enforce vs assert vs contracts and such, or whether such checks should or not stay or not in various distribution builds (mainly -release). I can see at least 5 cases, and am far to be sure what the proper tool is in every case, and in which builds it should stay. In each case, there is potential wrong input; but note the variety of cases does seems orthogonal (lol) to what kind of harm it may cause: * colleague: my code is called by code from the same app (same dev team); typically, wrong input logically cannot happen * friend: my code is called by code designed to cooperate with it; there is a kind of moral contract In both cases, wrong input reveals a bug; but in the first case, it's my own (team's) bug. I guess, but am not sure, these cases are good candidates for asserts (or contracts?), excluded from release build. * lib call: my code is a typical lib; thus, I have zero control on caller. I would let the check in release mode, thus use enforce. Or, use assert if it remains when the *caller* is compiled in debug mode. There is something unclear here, I guess. Maybe there are two sub-cases: ~ the caller logically should be able to prove its args correct ~ or not See, this is where I feel we have issues. The clear problem with *always* checking is the iota example. One may use iota like this: foreach(i; iota(0, 5)) Why should checks in iota remain for iota to prove that 0 is less than 5? It always will be less than 5, and the check is not necessary. [snip] This is the kind of job that the compiler could and should do. Whether it's assert and enforce, an inlining pass followed by value range propagation should simply eliminate the unnecessary tests. Andrei
Re: Against enforce()
On 03/18/2011 05:07 PM, Steven Schveighoffer wrote: On Fri, 18 Mar 2011 11:35:22 -0400, spir denis.s...@gmail.com wrote: On 03/18/2011 01:37 PM, Steven Schveighoffer wrote: This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. This is indeed a difficult topic. I'm a bit bluffed when reading people confidently asserting apparently clear positions about the use of enforce vs assert vs contracts and such, or whether such checks should or not stay or not in various distribution builds (mainly -release). I can see at least 5 cases, and am far to be sure what the proper tool is in every case, and in which builds it should stay. In each case, there is potential wrong input; but note the variety of cases does seems orthogonal (lol) to what kind of harm it may cause: * colleague: my code is called by code from the same app (same dev team); typically, wrong input logically cannot happen * friend: my code is called by code designed to cooperate with it; there is a kind of moral contract In both cases, wrong input reveals a bug; but in the first case, it's my own (team's) bug. I guess, but am not sure, these cases are good candidates for asserts (or contracts?), excluded from release build. * lib call: my code is a typical lib; thus, I have zero control on caller. I would let the check in release mode, thus use enforce. Or, use assert if it remains when the *caller* is compiled in debug mode. There is something unclear here, I guess. Maybe there are two sub-cases: ~ the caller logically should be able to prove its args correct ~ or not See, this is where I feel we have issues. The clear problem with *always* checking is the iota example. One may use iota like this: foreach(i; iota(0, 5)) Why should checks in iota remain for iota to prove that 0 is less than 5? It always will be less than 5, and the check is not necessary. checks should only be in place during release when the input to the function cannot be proven at compile time. When it can be proven, then the checks should go away. The problem I see is it's iota's responsibility to do those checks, but it has no idea where the data comes from. What I would suggest is to check at the point the argument data is created, not at the point where it's used. So for instance, if you get the parameters for iota from an input file, then you need to check those arguments before passing to iota. This is a difficult problem to solve, because one party knows whether the arguments need to be checked, and the other party knows how to check the arguments. I don't know if there is a clean way to do this. My thoughts are that phobos should only use enforce where it can prove the arguments are runtime-generated, and rely on asserts otherwise. The obvious pitfall is that one could pass runtime-generated data to a phobos function which uses asserts, and the program could crash on an otherwise recoverable error because the user of phobos did not validate the input first. I think the risk here is less important than the reduction in performance that occurs when enforce is used instead. Yes, I think you are correctly surrounding the issue. But my choice would rather be safety as default. Would you really let the func called by a/b non-check b!=0? Thus, I would consider allowing the caller stating don't check this call, not the opposite. Another issue is this creates one more complication in the language; and one orthogonal to the whole set of func-calls; with syntax needed, I guess. Another path is decision via compiler analysis: if arguments can be proved constant (I guess /this/ point can be made), then check is removed for release, automatically; else it cannot be removed at all. [Thanks for the precision about enforce vs alwaysAssert.] Denis -- _ vita es estrany spir.wikidot.com
Re: Against enforce()
On Fri, 18 Mar 2011 12:31:23 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 3/18/11 11:07 AM, Steven Schveighoffer wrote: On Fri, 18 Mar 2011 11:35:22 -0400, spir denis.s...@gmail.com wrote: On 03/18/2011 01:37 PM, Steven Schveighoffer wrote: This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. This is indeed a difficult topic. I'm a bit bluffed when reading people confidently asserting apparently clear positions about the use of enforce vs assert vs contracts and such, or whether such checks should or not stay or not in various distribution builds (mainly -release). I can see at least 5 cases, and am far to be sure what the proper tool is in every case, and in which builds it should stay. In each case, there is potential wrong input; but note the variety of cases does seems orthogonal (lol) to what kind of harm it may cause: * colleague: my code is called by code from the same app (same dev team); typically, wrong input logically cannot happen * friend: my code is called by code designed to cooperate with it; there is a kind of moral contract In both cases, wrong input reveals a bug; but in the first case, it's my own (team's) bug. I guess, but am not sure, these cases are good candidates for asserts (or contracts?), excluded from release build. * lib call: my code is a typical lib; thus, I have zero control on caller. I would let the check in release mode, thus use enforce. Or, use assert if it remains when the *caller* is compiled in debug mode. There is something unclear here, I guess. Maybe there are two sub-cases: ~ the caller logically should be able to prove its args correct ~ or not See, this is where I feel we have issues. The clear problem with *always* checking is the iota example. One may use iota like this: foreach(i; iota(0, 5)) Why should checks in iota remain for iota to prove that 0 is less than 5? It always will be less than 5, and the check is not necessary. [snip] This is the kind of job that the compiler could and should do. Whether it's assert and enforce, an inlining pass followed by value range propagation should simply eliminate the unnecessary tests. In this simple case yes, but inlining is not forceable, so you should not rely on it for optimizing out asserts or enforce. Inlining also only goes so deep, it doesn't make sense to inline a whole program, so at some point, you are going to use a function call, even though it can be proven the data is within range. From what I can see, we have 3 cases: 1. those where we can prove beyond a doubt that whether a value is valid is runtime dependent. Those cases should obviously use enforce. 2. Those where we can prove beyond a doubt that whether a value is valid does not depend on runtime data. Those cases should obviously use assert. 3. You can't prove beyond a doubt where those values come from. It's case 3 that is the troublesome one, not because it's hard to prove whether it's case 3 or not, but because the code that knows what the data could be (the caller) is separate from the code which knows whether its valid (the callee). It's also case 3 which is the most common. The most common case is a library function which wants to validate its input. So with the tools we have at hand today, which one should be applied to case 3? My preference is to use assert because then the caller has control over whether the data is checked or not. If you use enforce, the caller has no way of saying no really, I checked that value already!. Note also that phobos functions may be double checking data ALREADY if they make calls to each other. -Steve
Re: Against enforce()
On Friday, March 18, 2011 09:08:38 Kagamin wrote: Steven Schveighoffer Wrote: This is a good example of why it's difficult to decide what user input is. One could consider that the 'user' in this case is the developer using the library, but I don't think that's the right choice. I'd say it's a bug, this is clearly a contract, since the data being passed into the ctor can easily not be user input (i.e. it's most likely two literals that will never depend on a user). If it is user input, the caller of the ctor should enforce the user input before passing it to iota. You can't validate all user input, so external data ends up spead across your entire application. So I don't understand obsession with -release switch, because contracts most of the time do validate user input. If we think about -release switch as a HP-hack for exotic code, there will be no ideological difference between assert and enforce. The idea is that if an assertion fails, there is a bug in your program. You _cannot_ rely on an assertion being run, because it could be compiled out. It is _only_ for verifying that your code is correct. Exceptions, on the other hand, are used for handling errors. If something which results in an error but which is _not_ a logic bug in your program, then it should be handled by an exception. User input would be a classic example of this. If you get bad user input, that's not the program's fault. It can verify that the user input is valid and then assume that it's valid after that (at which point, using assert would make sense, because it's supposed to be guaranteed that the values are valid, since they were validated). But in handling the user input, it would throw on error, not assert. enforce is merely a shorthand way of testing and throw exceptions on failure. It makes exception handling look like assertions. But it is an _entirely_ different thing. As has been point out, the problem is in cases where it's not clear whether you should treat input as user input (and therefore needs to _always_ be checked and have exceptions thrown on error) or whether you should treat input as being from your program and guaranteed to be valid (at which point you use assert to check that that guarantee actually holds). Assertions are _not_ for error handling. They _kill_ your program on failure (since they throw an Assert_Error_, not an exception). Exceptions (and therefore enforce) are for error handling. There is a _clear_ and _distinct_ difference between the two. The confusion is not between assert and enforce. The confusion is when you have a situation where whether assert or enforce is appropriate depends on how the function is being used. - Jonathan M Davis
Re: Against enforce()
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 04:34:54 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer Wrote:
As long as the delegate does not access shared/global data, it
should be able to be pure. Even delegates which modify TLS data
should be able to be pure (weak-pure, but still pure).
TLS variables are global and must not be accessed from any function
marked as pure. With regard to purity, there isn't any difference
between shared and TLS variables.
However, it's still not shared.
This, for example, is a weak pure function:
void foo(int *n) pure { *n = 5;}
Because TLS variables are not shared, you should be able to do this:
int x;
void bar()
{
foo(x);
}
Yes, that compiles fine. But bar() is not pure.
But you are right, there is a huge difference between a local reference
to TLS data and directly accessing TLS data -- the latter can be
obscured from the compiler, resulting in the compiler thinking the
function can be strong pure.
So I don't know exactly how to mitigate this, but in my mind, it feels
like this should work:
int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
int x;
void bar()
{
foo(x == 4, x = 5);
}
It seems not too different from the above example where you pass the
address of x. But obviously the x = 5 delegate cannot be pure (it
modifies TLS data).
We may have no recourse to get this to work. It may be a lost cause,
and you just can't have lazy variables for pure functions.
It's not a lost cause, it's a two-liner!
mtype.c line 5045:
if (fparam-storageClass STClazy)
{
-error(0, cannot have lazy parameters to a pure
function);
+tf-purity = PUREweak;
+break;
}
This is a bit conservative: it would be possible to allow lazy
parameters to be marked as pure, which would allow them to be strongly
pure. But that would probably not be worth the extra complexity.
Re: Against enforce()
On Fri, 18 Mar 2011 14:35:27 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 04:34:54 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer Wrote:
As long as the delegate does not access shared/global data, it
should be able to be pure. Even delegates which modify TLS data
should be able to be pure (weak-pure, but still pure).
TLS variables are global and must not be accessed from any function
marked as pure. With regard to purity, there isn't any difference
between shared and TLS variables.
However, it's still not shared.
This, for example, is a weak pure function:
void foo(int *n) pure { *n = 5;}
Because TLS variables are not shared, you should be able to do this:
int x;
void bar()
{
foo(x);
}
Yes, that compiles fine. But bar() is not pure.
But you are right, there is a huge difference between a local
reference to TLS data and directly accessing TLS data -- the latter can
be obscured from the compiler, resulting in the compiler thinking the
function can be strong pure.
So I don't know exactly how to mitigate this, but in my mind, it feels
like this should work:
int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
int x;
void bar()
{
foo(x == 4, x = 5);
}
It seems not too different from the above example where you pass the
address of x. But obviously the x = 5 delegate cannot be pure (it
modifies TLS data).
We may have no recourse to get this to work. It may be a lost cause,
and you just can't have lazy variables for pure functions.
It's not a lost cause, it's a two-liner!
mtype.c line 5045:
if (fparam-storageClass STClazy)
{
-error(0, cannot have lazy parameters to a pure
function);
+tf-purity = PUREweak;
+break;
}
This is a bit conservative: it would be possible to allow lazy
parameters to be marked as pure, which would allow them to be strongly
pure. But that would probably not be worth the extra complexity.
I'm not sure this works. Aren't you allowed to pass in a delegate to a
lazy parameter?
For example:
shared int x;
int foo()
{
return x;
}
int bar(lazy int n) pure
{
return n;
}
void baz()
{
bar(foo);
}
or alternatively:
void baz()
{
bar(x);
}
The no-shared-data rule prevents you from passing in a shared int
reference to a pure function, but how do you stop a delegate from
accessing shared data?
-Steve
Re: Against enforce()
I would say it is a bug in the contract.
The signature is not normalized and the user gets a chance to provide
conflicting parameters. I think that it would be best to deduce the
direction
from the order of the start and end parameters. Then the stepsize can be
made
absolute.
Am 18.03.2011 09:14, schrieb Kagamin:
So this is a bug? This is a contract, not a validation of user input.
struct Iota(N, S) if ((isIntegral!N || isPointer!N) isIntegral!S)
{
private N current, pastLast;
private S step;
this(N current, N pastLast, S step)
{
enforce((current= pastLast step 0) ||
(current= pastLast step 0));
this.current = current;
this.step = step;
Re: Against enforce()
On Fri, 18 Mar 2011 11:02:13 +0100, Kagamin s...@here.lot wrote: Don Wrote: Pure functions calling weakly pure functions are also weakly pure and so on. This effectively leaves you without purity. I presume you mean Pure functions calling weakly pure functions *would also be* weakly pure and so on. ? What's the difference? Your original statement seems to indicate that strongly pure functions would be reduced to weakly pure if they call weakly pure functions. -- Simen
Re: Against enforce()
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 14:35:27 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer wrote:
On Fri, 18 Mar 2011 04:34:54 -0400, Don nos...@nospam.com wrote:
Steven Schveighoffer Wrote:
As long as the delegate does not access shared/global data, it
should be able to be pure. Even delegates which modify TLS data
should be able to be pure (weak-pure, but still pure).
TLS variables are global and must not be accessed from any function
marked as pure. With regard to purity, there isn't any difference
between shared and TLS variables.
However, it's still not shared.
This, for example, is a weak pure function:
void foo(int *n) pure { *n = 5;}
Because TLS variables are not shared, you should be able to do this:
int x;
void bar()
{
foo(x);
}
Yes, that compiles fine. But bar() is not pure.
But you are right, there is a huge difference between a local
reference to TLS data and directly accessing TLS data -- the latter
can be obscured from the compiler, resulting in the compiler thinking
the function can be strong pure.
So I don't know exactly how to mitigate this, but in my mind, it
feels like this should work:
int foo(bool cond, lazy int n) pure { if(cond) return n; return 0;}
int x;
void bar()
{
foo(x == 4, x = 5);
}
It seems not too different from the above example where you pass the
address of x. But obviously the x = 5 delegate cannot be pure (it
modifies TLS data).
We may have no recourse to get this to work. It may be a lost
cause, and you just can't have lazy variables for pure functions.
It's not a lost cause, it's a two-liner!
mtype.c line 5045:
if (fparam-storageClass STClazy)
{
-error(0, cannot have lazy parameters to a pure
function);
+tf-purity = PUREweak;
+break;
}
This is a bit conservative: it would be possible to allow lazy
parameters to be marked as pure, which would allow them to be strongly
pure. But that would probably not be worth the extra complexity.
I'm not sure this works. Aren't you allowed to pass in a delegate to a
lazy parameter?
Yes.
For example:
shared int x;
int foo()
{
return x;
}
int bar(lazy int n) pure
{
return n;
}
void baz()
{
bar(foo);
}
or alternatively:
void baz()
{
bar(x);
}
This compiles just fine. (Well, you need to use bar(foo) not bar(foo)).
But if you try to mark baz() as pure, here's what you get:
test0.d(135): Error: pure function 'baz' cannot call impure function 'foo'
or for the second case:
test0.d(136): Error: pure function 'baz' cannot access mutable static
data 'x'
bar is just weakly pure.
The no-shared-data rule prevents you from passing in a shared int
reference to a pure function, but how do you stop a delegate from
accessing shared data?
Delegates are either marked as pure, or not. In the case above, foo() is
not pure.
Re: Against enforce()
On Thu, 17 Mar 2011 03:03:23 +0100, Daniel Gibson metalcae...@gmail.com wrote: You're right. I just realized it and wanted to reply that correction to myself but you were faster :) I agree, that *is* helpful. A side note: The documentation says the value is returned if it's non-zero. enforce() also throws on null (for classes, probably also for pointers) and false (for bool..) - this should probably be mentioned in the docs ;-) null and false are also zero. :p -- Simen
Re: Against enforce()
bearophile Wrote: When you design a language and its standard library you have to keep a balance between making things very easy but not good enough, and very hard/fussy but better (see the simplicity of ddoc and D unittests, ddoc is mostly OK, but unnittests are probably a bit too much simple, they miss some essential features like a name). In my opinion a standard library is meant to throw a bit better exceptions than fully generic ones. .net framework was designed to use sizeable exception hierarchy, though this approach failed, in most cases everything goes ok and in the case of an error you can't recover, no matter what error it is. So the code usually ends up either catching Exception or not catching anything.
Re: Against enforce()
On Wed, 16 Mar 2011 21:07:54 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 03/16/2011 06:45 PM, bearophile wrote: - It kills inlining (with the current DMD, and I don't think this problem will be fixed soon); Not a problem of enforce. Why can't enforce be this: T enforce(T, string file = __FILE__, int line = __LINE__)(T value, /* lazy -- BUG: disables inlining */ const(char)[] msg = null); Until the compiler is fixed? I bet it would perform better even though the message may be eagerly constructed. Simply because in 99% of cases (I've checked) the message is a string literal. Essentially we keep saying D is on par with C for performance, and then the standard library is riddled with un-optimizable code making that claim patently false. It doesn't help to say, well yeah, I know it's not *now* but trust me, we know what the problem is and we'll fix it in the next 1-60 months. Given that this problem has been noted and existed for over a year, and doesn't show any sign of being fixed, we should work around it until it is fixed. enforce should be pure, since it should be equivalent to if(!x) throw new Exception(msg). However, we cannot have a function that is the equivalent. This would all be possible with macros. The workaround is simple, replace instances of enforce in functions you want to be pure with the equivalent if (!x) throw ... All the other points, I disagree with bearophile, enforce is not assert and should not be nothrow. -Steve
Re: Against enforce()
On Wed, 16 Mar 2011 22:10:01 -0400, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
On 03/16/2011 08:53 PM, bearophile wrote:
Andrei:
Yum, love the enforce.
You are silly :-)
So perhaps the language could be improved as enforce does not break
purity.
Currently enforce() uses a lazy argument. A lazy argument is a
delegate, and generally such delegate can't be pure, because the
expressions you give to enforce() usually refer to variables in the
scope where you call enforce(). So what kind of language improvements
are you thinking about?
I haven't thought through it. Clearly this is a false positive, a
restriction that should be at best removed.
int x;
void foo(int i)
{
enforce(i == 5, to!string(x = 5)); // if enforce is pure, this will
mean the global x could be affected inside the function
}
I don't think enforce can be straight pure. It can only be pure when
called from pure functions. A so-called conditional purity.
If we have auto purity, then I think the compiler could sort this out.
-Steve
Re: Against enforce()
Am 17.03.2011 06:59, schrieb Simen kjaeraas: On Thu, 17 Mar 2011 03:03:23 +0100, Daniel Gibson metalcae...@gmail.com wrote: You're right. I just realized it and wanted to reply that correction to myself but you were faster :) I agree, that *is* helpful. A side note: The documentation says the value is returned if it's non-zero. enforce() also throws on null (for classes, probably also for pointers) and false (for bool..) - this should probably be mentioned in the docs ;-) null and false are also zero. :p Technically yes, but it'd be more clear to specifically mention them. Also the value is returned could imply that enforce doesn't work with references (= objects).. I think enforce's behaviour on bools, objects and pointers would be easier to understand if null and false were mentioned :)
Re: Against enforce()
Steven Schveighoffer: All the other points, I disagree with bearophile, enforce is not assert and should not be nothrow. I have never said this. I am aware that enforce can't be nothrow, and I agree that in some situations you want to raise an exception instead of using asserts. One of the things I have said (maybe wrongly) are: You have to add that such if (!condition) throw new Exception(args) idiom is common in Phobos because Phobos is present only in release mode. If the zip distribution of DMD contains two Phobos and dmd becomes able to use the right one according to the compilation switches, then I think that if (!condition) throw new Exception(args) will become more rare, and the enforce() too will be less commonly needed. Bye, bearophile
Re: Against enforce()
On Thu, 17 Mar 2011 11:49:25 -0400, bearophile bearophileh...@lycos.com wrote: Steven Schveighoffer: All the other points, I disagree with bearophile, enforce is not assert and should not be nothrow. I have never said this. I am aware that enforce can't be nothrow, and I agree that in some situations you want to raise an exception instead of using asserts. One of the things I have said (maybe wrongly) are: You have to add that such if (!condition) throw new Exception(args) idiom is common in Phobos because Phobos is present only in release mode. If the zip distribution of DMD contains two Phobos and dmd becomes able to use the right one according to the compilation switches, then I think that if (!condition) throw new Exception(args) will become more rare, and the enforce() too will be less commonly needed. I was going off of this statement: It doesn't allow functions to be nothrow. This is a fault, because D has Contract Programming, that is meant to be usable for nothrow functions too. D Contracts with asserts are the right tool. Sorry if I misunderstood. But enforce is a simple factoring of the if(!condition) throw Exception(msg) into an expression. It is meant to throw an exception and meant to be used in release mode. The only problem I see with it is the inline-killing. -Steve
Re: Against enforce()
Steven Schveighoffer: The only problem I see with it is the inline-killing. Please don't ignore the purity-killing :-) Bye, bearophile
Re: Against enforce()
Steven Schveighoffer wrote: On Wed, 16 Mar 2011 21:07:54 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 03/16/2011 06:45 PM, bearophile wrote: - It kills inlining (with the current DMD, and I don't think this problem will be fixed soon); Not a problem of enforce. Why can't enforce be this: T enforce(T, string file = __FILE__, int line = __LINE__)(T value, /* lazy -- BUG: disables inlining */ const(char)[] msg = null); Until the compiler is fixed? I bet it would perform better even though the message may be eagerly constructed. Simply because in 99% of cases (I've checked) the message is a string literal. Essentially we keep saying D is on par with C for performance, and then the standard library is riddled with un-optimizable code making that claim patently false. It doesn't help to say, well yeah, I know it's not *now* but trust me, we know what the problem is and we'll fix it in the next 1-60 months. I completely agree. If we make statements like it's OK because it will be fixed soon it has to be added to a list and prioritized. With the understanding that *it knocks something else from that list*. As I've ended up being the one who implements a large fraction of such things, I get unhappy when people casually make statements about them being implemented soon.
Re: Against enforce()
bearophile Wrote:
Steven Schveighoffer:
The only problem I see with it is the inline-killing.
Please don't ignore the purity-killing :-)
void checkArgument(bool condition, const char[] msg = null, string file =
__FILE__, int line = __LINE__) pure
{
if(!condition)throw new ArgumentException(msg,file,line);
}
will this work?
Re: Against enforce()
Don: Ah, OK. I was pretty sure there wasn't a problem with delegates, didn't realize that lazy was the issue. Nonetheless, I think we can just overload enforce() to fix this problem. In one of the answers I've written to Andrei I have shown this link, it shows the problem is caused by lazy: http://d.puremagic.com/issues/show_bug.cgi?id=5746 What is the enforce() overload do you suggest to add to Phobos? Bye, bearophile
Re: Against enforce()
Kagamin: will this work? Maybe msg has to be of string type. Bye, bearophile
Re: Against enforce()
On Thu, 17 Mar 2011 12:09:50 -0400, bearophile bearophileh...@lycos.com wrote: Steven Schveighoffer: The only problem I see with it is the inline-killing. Please don't ignore the purity-killing :-) I think this is not as much an easy fix. By changing one line in enforce, every instance becomes inlinable. By making enforce also pure, it doesn't automatically make all users of enforce pure. I thought that lazy enforce cannot be pure, but I realize now that it can, as long as the delegate is pure. However, I think the compiler won't cooperate with that. If we temporarily disallow lazy and also make enforce pure, it will compile, but I'm worried one problem will be fixed and not the other, and then we are stuck with not being able to get the benefits of the lazy evaluation. I think the easiest thing to do right now is make enforce not lazy. That at least gets us inlining, which should be 90% of the performance problem with near-zero effort. If it makes sense in the future, we can also make it pure, or auto pure if that is ever supported. -Steve
Re: Against enforce()
On Thu, 17 Mar 2011 18:17:08 +0100, Steven Schveighoffer schvei...@yahoo.com wrote: On Thu, 17 Mar 2011 12:09:50 -0400, bearophile bearophileh...@lycos.com wrote: Steven Schveighoffer: The only problem I see with it is the inline-killing. Please don't ignore the purity-killing :-) I think this is not as much an easy fix. By changing one line in enforce, every instance becomes inlinable. By making enforce also pure, it doesn't automatically make all users of enforce pure. I thought that lazy enforce cannot be pure, but I realize now that it can, as long as the delegate is pure. However, I think the compiler won't cooperate with that. Not currently, at least. This made me wonder. A delegate created inside a pure function would have to be pure while in the scope of that function, right? Seems to me that should be possible to implement. -- Simen
Re: Against enforce()
On Thu, 17 Mar 2011 13:30:21 -0400, Simen kjaeraas simen.kja...@gmail.com wrote: On Thu, 17 Mar 2011 18:17:08 +0100, Steven Schveighoffer schvei...@yahoo.com wrote: On Thu, 17 Mar 2011 12:09:50 -0400, bearophile bearophileh...@lycos.com wrote: Steven Schveighoffer: The only problem I see with it is the inline-killing. Please don't ignore the purity-killing :-) I think this is not as much an easy fix. By changing one line in enforce, every instance becomes inlinable. By making enforce also pure, it doesn't automatically make all users of enforce pure. I thought that lazy enforce cannot be pure, but I realize now that it can, as long as the delegate is pure. However, I think the compiler won't cooperate with that. Not currently, at least. This made me wonder. A delegate created inside a pure function would have to be pure while in the scope of that function, right? Seems to me that should be possible to implement. As long as the delegate does not access shared/global data, it should be able to be pure. Even delegates which modify TLS data should be able to be pure (weak-pure, but still pure). This should be easy to enforce when the delegate is created automatically from an expression using a lazy call. However, we need some implicit casting rules for pure delegates into non-pure ones. -Steve
Re: Against enforce()
On 3/16/2011 6:07 PM, Andrei Alexandrescu wrote: On 03/16/2011 06:45 PM, bearophile wrote: So a better solution is for the standard Phobos library to ship in two versions, one compiled in release and not release mode, and DMD may choose the right one according to the compilation switches. This removes most of the need of enforce(). I suggest to deprecate enforce(). Until the problem with Phobos compilation is solved and enforces are removed from Phobos, enforce() may become a private Phobos function that user code can't import. There may be some confusion somewhere. enforce is not supposed to be a sort of assert. It is a different tool with a different charter. Use assert for assertions. I want to emphasize Andrei's point. 1. Asserts and contracts are for detecting program BUGS. They are not for validating user input, checking for disk full, file not found errors, etc. 2. Enforce is for validating user input, checking for disk full, file not found errors, etc. Enforce is NOT for use in contracts or checking for program bugs. Any use of enforce in Phobos that is checking for program bugs is itself a bug and should be entered into bugzilla for fixing.
alwaysAssert() [was: Against enforce()]
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
On 3/16/2011 6:07 PM, Andrei Alexandrescu wrote:
On 03/16/2011 06:45 PM, bearophile wrote:
So a better solution is for the standard Phobos library to ship in two
versions, one compiled in release and not release mode, and DMD may choose
the
right one according to the compilation switches. This removes most of the
need
of enforce(). I suggest to deprecate enforce(). Until the problem with
Phobos
compilation is solved and enforces are removed from Phobos, enforce() may
become a private Phobos function that user code can't import.
There may be some confusion somewhere. enforce is not supposed to be a sort
of
assert. It is a different tool with a different charter. Use assert for
assertions.
I want to emphasize Andrei's point.
1. Asserts and contracts are for detecting program BUGS. They are not for
validating user input, checking for disk full, file not found errors, etc.
2. Enforce is for validating user input, checking for disk full, file not
found
errors, etc. Enforce is NOT for use in contracts or checking for program bugs.
Any use of enforce in Phobos that is checking for program bugs is itself a bug
and should be entered into bugzilla for fixing.
I've asked for this before and I'll ask again: Can we **please** put an
alwaysAssert() function (or an abbreviation of this to make it less verbose) in
Phobos? I proposed this once before and it wasn't well liked for some reason
This reminded me to persist a little about it.
I sometimes abuse enforce() for non-performance critical asserts that I don't
want
to ever be turned off, but that are semantically asserts in that they're
supposed
to reveal bugs, not check for within-spec errors. I know this is The Wrong
Thing
to do, but it's too convenient and useful to stop doing it unless I have a good
alternative. The differences between enforce() and alwaysAssert() would be that
alwaysAssert() throws an AssertError instead of an Exception and that they
indicate different intents. alwaysAssert() could even be implemented in terms
of
enforce() using custom exceptions.
Example implementation:
void alwaysAssert(T, string file = __FILE__, string line = __LINE__)
(T value, lazy string msg = null) {
enforce!(T, file, line)(value, new AssertError(msg));
}
Re: Against enforce()
On 3/16/2011 6:53 PM, bearophile wrote: Right. But saying just that is not enough. You have to add that such if (!condition) throw new Exception(args) idiom is common in Phobos because Phobos is present only in release mode. If the zip distribution of DMD contains two Phobos and dmd becomes able to use the right one according to the compilation switches, then I think that if (!condition) throw new Exception(args) will become more rare, and the enforce() too will be less commonly needed. I must reiterate that enforce() is NOT FOR DETECTING PROGRAM BUGS. Therefore, enforce() must not change its behavior based on release mode or other compiler switches. Contracts and asserts are for program bug detection. NOT enforce. This distinction is critical.
Re: alwaysAssert() [was: Against enforce()]
On 3/17/2011 12:32 PM, dsimcha wrote: I've asked for this before and I'll ask again: Can we **please** put an alwaysAssert() function (or an abbreviation of this to make it less verbose) in Phobos? I proposed this once before and it wasn't well liked for some reason This reminded me to persist a little about it. Not always the prettiest, but you can write: if (!condition) assert(0); The assert(0) will be replaced with a HLT instruction even in release mode.
Re: alwaysAssert() [was: Against enforce()]
== Quote from Walter Bright (newshou...@digitalmars.com)'s article On 3/17/2011 12:32 PM, dsimcha wrote: I've asked for this before and I'll ask again: Can we **please** put an alwaysAssert() function (or an abbreviation of this to make it less verbose) in Phobos? I proposed this once before and it wasn't well liked for some reason This reminded me to persist a little about it. Not always the prettiest, but you can write: if (!condition) assert(0); The assert(0) will be replaced with a HLT instruction even in release mode. This is better than nothing, but not enough to convince me to stop abusing enforce(). I much prefer to have a real alwaysAssert() function that gives a file, a line number and an error message.
Re: Against enforce()
On Thursday, March 17, 2011 12:22:00 Walter Bright wrote: On 3/16/2011 6:53 PM, bearophile wrote: Right. But saying just that is not enough. You have to add that such if (!condition) throw new Exception(args) idiom is common in Phobos because Phobos is present only in release mode. If the zip distribution of DMD contains two Phobos and dmd becomes able to use the right one according to the compilation switches, then I think that if (!condition) throw new Exception(args) will become more rare, and the enforce() too will be less commonly needed. I must reiterate that enforce() is NOT FOR DETECTING PROGRAM BUGS. Therefore, enforce() must not change its behavior based on release mode or other compiler switches. Contracts and asserts are for program bug detection. NOT enforce. This distinction is critical. I completely agree with you. However, I think that part of the confusion is that there was discussion of using enforce in Phobos in some cases where we might otherwise have used an assertion, because the assertions would general be compiled out when anyone went to use Phobos other than Phobos itself, so they would be useless. I'm not aware that ever actually having been done, however. And in general, I don't like the idea of using assertions to validate that someone is using a library function correctly rather than validating the library function itself. I really think that that should be treated like user input and throw an exception if it really is supposed to be being validated. Some additional assertions which could be of benefit both in unit testing and if some actually uses a non-release version of Phobos might be useful, but counting on that sort of check being there makes no sense to me. Assertions are for validating the code that they're in, not someone else's code which is using that code. In any case, I think that part of the confusion here is due to previous discussions on the lack of assertions in -release (and the fact that libphobos.a is normally compiled with -release) and the possible use of enforce to get around that in some cases. I'm not aware that actually ever having been done, however. Still, I do get the impression the people often confuse the purposes of assertions and exceptions. - Jonathan M Davis
Re: alwaysAssert() [was: Against enforce()]
On 3/17/2011 1:49 PM, dsimcha wrote:
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
On 3/17/2011 12:32 PM, dsimcha wrote:
I've asked for this before and I'll ask again: Can we **please** put an
alwaysAssert() function (or an abbreviation of this to make it less verbose) in
Phobos? I proposed this once before and it wasn't well liked for some reason
This reminded me to persist a little about it.
Not always the prettiest, but you can write:
if (!condition) assert(0);
The assert(0) will be replaced with a HLT instruction even in release mode.
This is better than nothing, but not enough to convince me to stop abusing
enforce(). I much prefer to have a real alwaysAssert() function that gives a
file, a line number and an error message.
if (!condition) { writeln(my message %s %s, __FILE__, __LINE__); assert(0); }
I am strongly opposed to using enforce() for bug detection.
Re: alwaysAssert() [was: Against enforce()]
On 03/17/2011 05:03 PM, Walter Bright wrote:
On 3/17/2011 1:49 PM, dsimcha wrote:
== Quote from Walter Bright (newshou...@digitalmars.com)'s article
On 3/17/2011 12:32 PM, dsimcha wrote:
I've asked for this before and I'll ask again: Can we **please** put an
alwaysAssert() function (or an abbreviation of this to make it less
verbose) in
Phobos? I proposed this once before and it wasn't well liked for
some reason
This reminded me to persist a little about it.
Not always the prettiest, but you can write:
if (!condition) assert(0);
The assert(0) will be replaced with a HLT instruction even in release
mode.
This is better than nothing, but not enough to convince me to stop
abusing
enforce(). I much prefer to have a real alwaysAssert() function that
gives a
file, a line number and an error message.
if (!condition) { writeln(my message %s %s, __FILE__, __LINE__);
assert(0); }
I am strongly opposed to using enforce() for bug detection.
There's nothing wrong with encapsulating this idiom (if frequent).
That's what essentially alwaysAssert does. So you're right but without
contradicting dsimcha's suggestion.
Andrei
Re: alwaysAssert() [was: Against enforce()]
On 3/17/2011 3:08 PM, Andrei Alexandrescu wrote:
if (!condition) { writeln(my message %s %s, __FILE__, __LINE__);
assert(0); }
I am strongly opposed to using enforce() for bug detection.
There's nothing wrong with encapsulating this idiom (if frequent). That's what
essentially alwaysAssert does. So you're right but without contradicting
dsimcha's suggestion.
Sure, but there is plenty wrong with using enforce() for bug detection even if
alwaysAssert does not exist. For one thing, such uses encourage others to
misunderstand and misuse enforce.
Additionally, alwaysAssert is an obvious one liner. I think such things need to
be very frequently used to consider them part of the standard library.
Otherwise, we risk Phobos becoming a morass of trivia.
Re: alwaysAssert() [was: Against enforce()]
On 3/17/2011 7:12 PM, Walter Bright wrote: Sure, but there is plenty wrong with using enforce() for bug detection even if alwaysAssert does not exist. For one thing, such uses encourage others to misunderstand and misuse enforce. Additionally, alwaysAssert is an obvious one liner. I think such things need to be very frequently used to consider them part of the standard library. Otherwise, we risk Phobos becoming a morass of trivia. What makes you think it wouldn't be used very frequently? It seems silly to me to turn off asserts in non-performance-critical bits of code just because I don't want bounds checking or the more expensive asserts.
Re: alwaysAssert() [was: Against enforce()]
On 03/17/2011 06:12 PM, Walter Bright wrote:
On 3/17/2011 3:08 PM, Andrei Alexandrescu wrote:
if (!condition) { writeln(my message %s %s, __FILE__, __LINE__);
assert(0); }
I am strongly opposed to using enforce() for bug detection.
There's nothing wrong with encapsulating this idiom (if frequent).
That's what
essentially alwaysAssert does. So you're right but without contradicting
dsimcha's suggestion.
Sure, but there is plenty wrong with using enforce() for bug detection
even if alwaysAssert does not exist. For one thing, such uses encourage
others to misunderstand and misuse enforce.
(I'm not sure what you even want to convey here as your statement does
not contradict or oppose any other, yet is grammatically formulated as
if it did. One may as well reply with Yes, but the water's wet.)
No question enforce would be wrong for bug detection. Using this would
be just as wrong:
if (undefinedState) throw new Exception(Hmm, odd...);
By packaging the if/throw combo, enforce becomes a notion, a part of the
vocabulary, a meme if you wish. That makes it neither more nor less
wrong for bug detection, but makes it plenty better for systematic error
handling.
Now, there is an additional issue that may confuse people on the
relative roles of assert and enforce. We have faced the question, how do
we validate arguments to a function in Phobos?
Initially I used enforce() all over the place, under the assumption that
Phobos' user and Phobos are separate entities. Therefore, any argument
coming from outside Phobos would be necessarily considered I/O from
within Phobos and therefore scrubbed accordingly.
Then there was pressure on loss of efficiency due to checking, so I and
others replaced certain instances of enforce() with assert(). This is
not wrong at all. Instead, it reflects a different view of the dynamics
between Phobos and its user: now Phobos + user code is considered as an
entity that works together. So a failing assert in Phobos could describe
a bug in Phobos itself or one in the application code that uses it. This
is more ambiguous but not one bit less correct under the stated assumptions.
Additionally, alwaysAssert is an obvious one liner. I think such things
need to be very frequently used to consider them part of the standard
library. Otherwise, we risk Phobos becoming a morass of trivia.
enforce() is an obvious one liner too. Size is not the only means to
assess the usefulness of an abstraction. Two others are frequency of the
pattern (think writeln) and ascribing a meaningful name to it (think
unittest).
http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-12.html#%_sec_1.3
is a good read, let me paste:
=
We have seen that procedures are, in effect, abstractions that describe
compound operations on numbers independent of the particular numbers.
For example, when we
(define (cube x) (* x x x))
we are not talking about the cube of a particular number, but rather
about a method for obtaining the cube of any number. Of course we could
get along without ever defining this procedure, by always writing
expressions such as
(* 3 3 3)
(* x x x)
(* y y y)
and never mentioning cube explicitly. This would place us at a serious
disadvantage, forcing us to work always at the level of the particular
operations that happen to be primitives in the language (multiplication,
in this case) rather than in terms of higher-level operations. Our
programs would be able to compute cubes, but our language would lack the
ability to express the concept of cubing. One of the things we should
demand from a powerful programming language is the ability to build
abstractions by assigning names to common patterns and then to work in
terms of the abstractions directly. Procedures provide this ability.
This is why all but the most primitive programming languages include
mechanisms for defining procedures.
=
Andrei
Re: alwaysAssert() [was: Against enforce()]
On 03/17/2011 06:38 PM, dsimcha wrote: On 3/17/2011 7:12 PM, Walter Bright wrote: Sure, but there is plenty wrong with using enforce() for bug detection even if alwaysAssert does not exist. For one thing, such uses encourage others to misunderstand and misuse enforce. Additionally, alwaysAssert is an obvious one liner. I think such things need to be very frequently used to consider them part of the standard library. Otherwise, we risk Phobos becoming a morass of trivia. What makes you think it wouldn't be used very frequently? It seems silly to me to turn off asserts in non-performance-critical bits of code just because I don't want bounds checking or the more expensive asserts. To me it sounds perfectly normal that there are integrity checks that you want to keep at all times, whereas others you only want to keep in a debug build. Andrei
Re: alwaysAssert() [was: Against enforce()]
On 3/17/2011 4:38 PM, dsimcha wrote:
On 3/17/2011 7:12 PM, Walter Bright wrote:
Sure, but there is plenty wrong with using enforce() for bug detection
even if alwaysAssert does not exist. For one thing, such uses encourage
others to misunderstand and misuse enforce.
Additionally, alwaysAssert is an obvious one liner. I think such things
need to be very frequently used to consider them part of the standard
library. Otherwise, we risk Phobos becoming a morass of trivia.
What makes you think it wouldn't be used very frequently?
I don't know if it would or would not be used very frequently.
It seems silly to me
to turn off asserts in non-performance-critical bits of code just because I
don't want bounds checking or the more expensive asserts.
The use case is more constrained than that. Because of the existence of:
if (!condition) assert(0);
the alwaysAssert is constrained to those purposes where the user feels the need
for file/line/message for bug detection in released code, and also does not want
to use:
if (!condition) { writeln(my message %s %s, __FILE__, __LINE__);
assert(0); }
Furthermore, nothing prevents the user from writing his own alwaysAssert.
For inclusion in Phobos, the more trivial something is, the higher utility it
needs to have to justify it. Of course, there's an element of subjectivity to
these judgments. One liners, though, should always be subject to significant
scrutiny and justification. Once in, we'll be stuck with them for a long time.
Re: alwaysAssert() [was: Against enforce()]
On 3/17/2011 5:02 PM, Andrei Alexandrescu wrote: [...] I don't disagree with anything you wrote. But I am suggesting that one liners should have a high utility to be justifiably included in Phobos. - You mentioned wondering where we should draw the line in using asserts to check function inputs as opposed to using enforce. I suggest that line should be when a shared library/dll boundary is crossed. Statically linked libs should use assert. The reason is straightforward - a shared library/dll cannot know in advance what will be connected to it, so it should treat data coming in from an external source as untrusted input. A statically linked library, on the other hand, is inextricably bound to a specific caller and is debugged/tested as a whole. This raises the spectre about what to do with Phobos if Phobos is built as a dll.
Re: alwaysAssert() [was: Against enforce()]
On 3/17/11 7:16 PM, Walter Bright wrote:
On 3/17/2011 4:38 PM, dsimcha wrote:
On 3/17/2011 7:12 PM, Walter Bright wrote:
Sure, but there is plenty wrong with using enforce() for bug detection
even if alwaysAssert does not exist. For one thing, such uses encourage
others to misunderstand and misuse enforce.
Additionally, alwaysAssert is an obvious one liner. I think such things
need to be very frequently used to consider them part of the standard
library. Otherwise, we risk Phobos becoming a morass of trivia.
What makes you think it wouldn't be used very frequently?
I don't know if it would or would not be used very frequently.
It seems silly to me
to turn off asserts in non-performance-critical bits of code just
because I
don't want bounds checking or the more expensive asserts.
The use case is more constrained than that. Because of the existence of:
if (!condition) assert(0);
the alwaysAssert is constrained to those purposes where the user feels
the need for file/line/message for bug detection in released code, and
also does not want to use:
if (!condition) { writeln(my message %s %s, __FILE__, __LINE__);
assert(0); }
Note that you have (twice in two different posts) a bug in your code:
you should have used writefln, not writeln. This humorously ruins your
point, particularly because it's a bug likely to make it all the way to
production (since that code path would normally not be exercised).
Furthermore, the format of the message is decided with every call
instead of centrally.
Furthermore, nothing prevents the user from writing his own alwaysAssert.
Conversely, nothing prevents the library from defining a function if it
deems it widely useful, even if short.
For inclusion in Phobos, the more trivial something is, the higher
utility it needs to have to justify it. Of course, there's an element of
subjectivity to these judgments. One liners, though, should always be
subject to significant scrutiny and justification. Once in, we'll be
stuck with them for a long time.
Correct. There are also other criteria such as standardization.
Primitives for logging would score strongly on the standardization
scale, though they are often trivial to implement.
You rose things debug and unittest all the way up to coveted keyword
status because you correctly understood that, although such items could
have easily been left to the implementation, everybody would have chosen
their own devices creating a morass of incompatibility. I find it
incongruent that you so strongly believe such success cannot be reproduced.
Andrei
Re: alwaysAssert() [was: Against enforce()]
On 3/17/11 7:31 PM, Walter Bright wrote: On 3/17/2011 5:02 PM, Andrei Alexandrescu wrote: [...] I don't disagree with anything you wrote. But I am suggesting that one liners should have a high utility to be justifiably included in Phobos. - You mentioned wondering where we should draw the line in using asserts to check function inputs as opposed to using enforce. I suggest that line should be when a shared library/dll boundary is crossed. Statically linked libs should use assert. The reason is straightforward - a shared library/dll cannot know in advance what will be connected to it, so it should treat data coming in from an external source as untrusted input. A statically linked library, on the other hand, is inextricably bound to a specific caller and is debugged/tested as a whole. This raises the spectre about what to do with Phobos if Phobos is built as a dll. These are all very good points and insights. We should keep them in mind. Andrei
Re: alwaysAssert() [was: Against enforce()]
On 2011-03-17 20:31:43 -0400, Walter Bright newshou...@digitalmars.com said:
On 3/17/2011 5:02 PM, Andrei Alexandrescu wrote:
[...]
I don't disagree with anything you wrote. But I am suggesting that one
liners should have a high utility to be justifiably included in Phobos.
-
You mentioned wondering where we should draw the line in using asserts
to check function inputs as opposed to using enforce. I suggest that
line should be when a shared library/dll boundary is crossed.
Statically linked libs should use assert.
The reason is straightforward - a shared library/dll cannot know in
advance what will be connected to it, so it should treat data coming in
from an external source as untrusted input. A statically linked
library, on the other hand, is inextricably bound to a specific caller
and is debugged/tested as a whole.
This raises the spectre about what to do with Phobos if Phobos is built
as a dll.
This would be much easier to work with if the decision about checking
in contracts was taken at the call site. If user code is compiled
with contracts, any user code calling Phobos would check the contracts
too, dynamic library or not.
One way to make this work is by making the compiler take the contract
as a separate function to call. For instance, this function:
int test(int i)
in {
assert(i = 0);
}
body {
return 100 / i;
}
would be split in two:
int testContract(int i)
{
assert(i = 0);
return test(i); // could hard-code tail call optimization here
}
int test(int i) {
return 100 / i;
}
If the function that calls this test function is compiled with
contracts turned on, it substitutes the call to test() for a call to
testContract(), and the contracts gets checked. The testContract()
function does not necessarily need to be part of the library, it can be
instantiated as needed at the call point.
It could even work with virtual functions: if test() is virtual,
testContract() would remain non-virtual and a call to the virtual
function test() would be replaced with a call to the non-virtual
function testContract() (which would be charged to call test).
There's two concerns however:
1. the contract would be checked against the static type instead of the
dynamic type as it is right now. Checking against the dynamic type
would require a new slot on the vtable -- or a separate vtable, perhaps
inside of the ClassInfo -- which would require contracts for public and
protected functions to be part of the library at all times.
2. taking the address of a function (or a delegate) could give you the
one that includes the contract if contracts are turned on, but that
would mean that code compiled with contracts and code compiled without
them would get different addresses for the same function inside of the
same executable, which could break some expectations.
--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/
Re: alwaysAssert() [was: Against enforce()]
On Thursday, March 17, 2011 18:23:51 Andrei Alexandrescu wrote: On 3/17/11 7:31 PM, Walter Bright wrote: On 3/17/2011 5:02 PM, Andrei Alexandrescu wrote: [...] I don't disagree with anything you wrote. But I am suggesting that one liners should have a high utility to be justifiably included in Phobos. - You mentioned wondering where we should draw the line in using asserts to check function inputs as opposed to using enforce. I suggest that line should be when a shared library/dll boundary is crossed. Statically linked libs should use assert. The reason is straightforward - a shared library/dll cannot know in advance what will be connected to it, so it should treat data coming in from an external source as untrusted input. A statically linked library, on the other hand, is inextricably bound to a specific caller and is debugged/tested as a whole. This raises the spectre about what to do with Phobos if Phobos is built as a dll. These are all very good points and insights. We should keep them in mind. It also brings us back to one of Bearophile's concerns. If you want assertions to be enabled in Phobos, you need a non-release build, and Phobos is distributed in non-release build. The only way to have a non-release build is to build it yourself and replace your libphobos.a (or phobos.lib) with the non-release version. Some folks want a way to make it so that there's a release and non- release version of Phobos and dmd picks the release version when you compile with -release and picks the non-release version otherwise. - Jonathan M Davis
Re: alwaysAssert() [was: Against enforce()]
On 3/17/2011 5:41 PM, Andrei Alexandrescu wrote:
if (!condition) { writeln(my message %s %s, __FILE__, __LINE__);
assert(0); }
Note that you have (twice in two different posts)
That's because I used cut paste.
a bug in your code: you should have used writefln, not writeln.
I made a mistake, I should have tested it.
This humorously ruins your point,
Come on. In any case, it would print:
my message %s %stest.d6
core.exception.AssertError@test(6): Assertion failure
and the information is still there, just badly formatted.
Furthermore, nothing prevents the user from writing his own alwaysAssert.
Conversely, nothing prevents the library from defining a function if it deems it
widely useful, even if short.
Right. So is it so widely useful that it justifies the cognitive load of
documentation of it and filling up Phobos with trivial stuff? It's not just
alwaysAssert(), this particular issue comes up again and again. The last time it
was for a suite of math functions that took degrees instead of radians.
I think what we need in Phobos is nontrivial stuff. Things like the fast xml
library, network code, database interface, etc.
For inclusion in Phobos, the more trivial something is, the higher
utility it needs to have to justify it. Of course, there's an element of
subjectivity to these judgments. One liners, though, should always be
subject to significant scrutiny and justification. Once in, we'll be
stuck with them for a long time.
Correct. There are also other criteria such as standardization. Primitives for
logging would score strongly on the standardization scale, though they are often
trivial to implement.
You rose things debug and unittest all the way up to coveted keyword status
because you correctly understood that, although such items could have easily
been left to the implementation, everybody would have chosen their own devices
creating a morass of incompatibility. I find it incongruent that you so strongly
believe such success cannot be reproduced.
There's a limited amount of such things one should do. Where does it stop?
Each addition must be judged on its own merits, not on the merits of something
else. As many have pointed out, D does not follow any of its principles 100%,
because doing so will drive it into a ditch, as do all things that value
adherence to principle over utility.
Against enforce()
enforce() seems the part of Phobos that I hate more. Among its faults: - It can't be used in weakly pure functions (because you can't use lazy arguments in pure functions), and not-purity is viral. Lot of Phobos can't be pure because of this, so even more user code can't be pure because of this; - It kills inlining (with the current DMD, and I don't think this problem will be fixed soon); - I have seen it slow down code (probably mostly because of its not-inlining nature); - Where it is used it usually doesn't give a more meaningful exception like WrongArgumentException, etc. I don't want a deep hierarchy of one hundred standard exceptions, but I think some standard exceptions for the most common mistakes, like wrong arguments, etc, are better than a generic enforce(), especially for a standard library code that is meant to be written with care and to give better error messages/exceptions. - It doesn't allow functions to be nothrow. This is a fault, because D has Contract Programming, that is meant to be usable for nothrow functions too. D Contracts with asserts are the right tool. I see enforce() just as a temporary workaround for a problem of Phobos (that it's compiled in release mode, so its asserts are vanished) that risks to become a permanent part of Phobos. So a better solution is for the standard Phobos library to ship in two versions, one compiled in release and not release mode, and DMD may choose the right one according to the compilation switches. This removes most of the need of enforce(). I suggest to deprecate enforce(). Until the problem with Phobos compilation is solved and enforces are removed from Phobos, enforce() may become a private Phobos function that user code can't import. Bye, bearophile
Re: Against enforce()
bearophile wrote: - It doesn't allow functions to be nothrow. This is a fault, because D has Contract Programming, that is meant to be usable for nothrow functions too. D Contracts with asserts are the right tool. assert and enforce cover a completely different problem, and aren't interchangeable. assert catches programming errors. If an assert fails, it's a bug that the programmer should fix. enforce, on the other hand, catches runtime errors that aren't the programmer's fault. Then *have* to be handled to be correct, so being nothrow is out of the question. The function is *not* guaranteed to succeed at runtime. Look at the way enforce is generally used inside Phobos. enforce(fp = fopen(file, r)); That is something that could fail at no fault of the programmer. It's an exception, not an assert. By far, the majority of enforces in Phobos are of this variety. They cannot possibly be made into contracts, and even if they could, that would be wrong. That said, I tentatively agree that enforce may be bad right now because of the other things you said (except the meaningful exception. enforce does that with an optional argument.) enforce is just a lazy way to write if(x) throw T(msg); It is elegant, it is beautiful, but if the practical problems aren't fixed, it might be right to avoid it anyway. But it'd have to be replaced with if/throw, not assert.
Re: Against enforce()
On Wednesday, March 16, 2011 16:45:56 bearophile wrote: enforce() seems the part of Phobos that I hate more. Among its faults: - It can't be used in weakly pure functions (because you can't use lazy arguments in pure functions), and not-purity is viral. Lot of Phobos can't be pure because of this, so even more user code can't be pure because of this; - It kills inlining (with the current DMD, and I don't think this problem will be fixed soon); - I have seen it slow down code (probably mostly because of its not-inlining nature); - Where it is used it usually doesn't give a more meaningful exception like WrongArgumentException, etc. I don't want a deep hierarchy of one hundred standard exceptions, but I think some standard exceptions for the most common mistakes, like wrong arguments, etc, are better than a generic enforce(), especially for a standard library code that is meant to be written with care and to give better error messages/exceptions. A number of modules in Phobos have exceptions specific to that module - such as DateTimeException or FileException. enforce doesn't stop anyone from using specific exceptions. And it's just as easy to use Exception instead of a specific exception type when throwing directly, so I don't think that this complaint holds much water. It's a valid complaint if Phobos developers are choosing to throw Exception instead of more specific exceptions, but that's not enforce's fault. - It doesn't allow functions to be nothrow. This is a fault, because D has Contract Programming, that is meant to be usable for nothrow functions too. D Contracts with asserts are the right tool. Overall, I favor exceptions when it comes to testing input rather than assertions, but regardless of that. enforce does _not_ stop functions from being nothrow. It just makes it more annoying to make them nothrow. If you want the caller to be nothrow, you have to have a try-catch block in the caller which catches Exception. If the function really isn't ever going to throw, then you can use such a try-catch block with an assert(0) in the catch body, and it's quite safe. std.datetime does that in several places. Regardless, your problem here really isn't enforce anyway. Your problem is that you think that a number of functions should be using assertions instead of exceptions. enforce happens to be a way to throw exceptions, but it's quite easy to use exceptions without it. So, enforce doesn't really have anything to do with it other than the fact that that was the means used to throw the exception. I see enforce() just as a temporary workaround for a problem of Phobos (that it's compiled in release mode, so its asserts are vanished) that risks to become a permanent part of Phobos. So a better solution is for the standard Phobos library to ship in two versions, one compiled in release and not release mode, and DMD may choose the right one according to the compilation switches. This removes most of the need of enforce(). I suggest to deprecate enforce(). Until the problem with Phobos compilation is solved and enforces are removed from Phobos, enforce() may become a private Phobos function that user code can't import. There a number of places in Phobos which throw exceptions and _should_ throw exceptions. ConvExceptions and FileExceptions are great examples. DateTimeException is used liberally in std.datetime, and it really wouldn't make sense to make them use assertions. And even in cases where exceptions were chosen of assertions because of the release vs non-release mode issue, there's still the question of whether we really want Phobos to be throwing AssertErrors, since when a programmer sees an assertion which isn't theirs, they're likely to think that it's someone else's code which is broken, not theirs. So, having AssertErrors thrown from Phobos look just plain bad. Now, in same cases performance still makes assertions more desirable - and there _are_ places in Phobos which use exceptions - but there are still a lot of situations where Phobos _should_ be throwing exceptions. Regardless, enforce is supposed to make throwing exceptions similar to asserting something. It is - in theory at least - a great idea. The problem with it is its laziness and all of the implications that that has (like the inability to inline because of it). If we had a non-lazy version of enforce or if the compiler were made smart enough to realize that it didn't need a lazy version in some cases (such as when it's passed a string literal) and to use a non-lazy version in such cases, then the problem would be reduced. Personally, I rarely use enforce precisely because of the inlining issue. I don't that that if(!condition) throw new ExceptionType(msg); is really much worse than enforce(condition, new ExceptionType(msg)); but I see no reason to get rid enforce. I also disagree with you about how much Phobos should be using exceptions. There _are_
Re: Against enforce()
On 03/16/2011 06:45 PM, bearophile wrote: enforce() seems the part of Phobos that I hate more. Yum, love the enforce. Among its faults: - It can't be used in weakly pure functions (because you can't use lazy arguments in pure functions), and not-purity is viral. Lot of Phobos can't be pure because of this, so even more user code can't be pure because of this; So perhaps the language could be improved as enforce does not break purity. - It kills inlining (with the current DMD, and I don't think this problem will be fixed soon); Not a problem of enforce. - I have seen it slow down code (probably mostly because of its not-inlining nature); Not a problem of enforce. - Where it is used it usually doesn't give a more meaningful exception like WrongArgumentException, etc. I don't want a deep hierarchy of one hundred standard exceptions, but I think some standard exceptions for the most common mistakes, like wrong arguments, etc, are better than a generic enforce(), especially for a standard library code that is meant to be written with care and to give better error messages/exceptions. enforce helps such idioms, does not prevent them. From the docs: === T enforce(T)(T value, lazy Throwable ex); If value is nonzero, returns it. Otherwise, throws ex. === - It doesn't allow functions to be nothrow. This is a fault, because D has Contract Programming, that is meant to be usable for nothrow functions too. D Contracts with asserts are the right tool. enforce's specification specifies it throws. It would therefore be difficult for it to not throw. This complaint is non sequitur. I see enforce() just as a temporary workaround for a problem of Phobos (that it's compiled in release mode, so its asserts are vanished) that risks to become a permanent part of Phobos. enforce is a simple abstraction of the idiom if (!condition) throw new Exception(args). If that idiom were rare, then occurrences of enforce would be rare and therefore there would be little need to have enforce at all. So a better solution is for the standard Phobos library to ship in two versions, one compiled in release and not release mode, and DMD may choose the right one according to the compilation switches. This removes most of the need of enforce(). I suggest to deprecate enforce(). Until the problem with Phobos compilation is solved and enforces are removed from Phobos, enforce() may become a private Phobos function that user code can't import. There may be some confusion somewhere. enforce is not supposed to be a sort of assert. It is a different tool with a different charter. Use assert for assertions. Andrei
Re: Against enforce()
Andrei: Yum, love the enforce. You are silly :-) So perhaps the language could be improved as enforce does not break purity. Currently enforce() uses a lazy argument. A lazy argument is a delegate, and generally such delegate can't be pure, because the expressions you give to enforce() usually refer to variables in the scope where you call enforce(). So what kind of language improvements are you thinking about? This is an example: http://d.puremagic.com/issues/show_bug.cgi?id=5746 - It kills inlining (with the current DMD, and I don't think this problem will be fixed soon); Not a problem of enforce. In my opinion it's not wise use widely in the standard library something that requires an optimization that the DMD compiler is not going to have soon, and that makes the code significantly slower. In some cases this almost forces me to keep a patched version of Phobos, or to add more nearly duplicated functions to my dlibs2. enforce helps such idioms, does not prevent them. From the docs: === T enforce(T)(T value, lazy Throwable ex); If value is nonzero, returns it. Otherwise, throws ex. === Then why is iota() using a nude enforce() instead of the enforce() with a more meaningful exception like WrongArgumentException? I have seen the nude enforce used in other parts of Phobos. So maybe enforce() makes even the standard library writers lazy. enforce's specification specifies it throws. It would therefore be difficult for it to not throw. This complaint is non sequitur. I know, and I agree in some situations you want an assert (to test arguments coming from outside), so in some situations an enforce is acceptable. The problem is that currently enforce is used as a patch for a DMD/Phobos problem that I hope will be fixed in a short time, see below. I see enforce() just as a temporary workaround for a problem of Phobos (that it's compiled in release mode, so its asserts are vanished) that risks to become a permanent part of Phobos. enforce is a simple abstraction of the idiom if (!condition) throw new Exception(args). If that idiom were rare, then occurrences of enforce would be rare and therefore there would be little need to have enforce at all. There may be some confusion somewhere. enforce is not supposed to be a sort of assert. It is a different tool with a different charter. Use assert for assertions. Right. But saying just that is not enough. You have to add that such if (!condition) throw new Exception(args) idiom is common in Phobos because Phobos is present only in release mode. If the zip distribution of DMD contains two Phobos and dmd becomes able to use the right one according to the compilation switches, then I think that if (!condition) throw new Exception(args) will become more rare, and the enforce() too will be less commonly needed. Bye, bearophile
Re: Against enforce()
Am 17.03.2011 02:07, schrieb Andrei Alexandrescu: On 03/16/2011 06:45 PM, bearophile wrote: - Where it is used it usually doesn't give a more meaningful exception like WrongArgumentException, etc. I don't want a deep hierarchy of one hundred standard exceptions, but I think some standard exceptions for the most common mistakes, like wrong arguments, etc, are better than a generic enforce(), especially for a standard library code that is meant to be written with care and to give better error messages/exceptions. enforce helps such idioms, does not prevent them. From the docs: === T enforce(T)(T value, lazy Throwable ex); If value is nonzero, returns it. Otherwise, throws ex. === Really? using enforce with a custom throwable saves *one* char: enforce(foo, new BlaException(bad!)); if(!foo) throw new BlaException(bad!); or are there other merits? But using enforce with a custom message (thus it'll just throw a standard Exception with that message), like assert, really shortens things: enforce(..., bad!); if(!...) throw new Exception(bad!); So of course enforce allows to throw meaningful exceptions, but it doesn't make it considerably easier/shorter. Cheers, - Daniel
Re: Against enforce()
I know, and I agree in some situations you want an assert ( I meant: I know, and I agree in some situations you want an exception ( Sorry, bearophile
Re: Against enforce()
On 03/16/2011 08:50 PM, Daniel Gibson wrote: Am 17.03.2011 02:07, schrieb Andrei Alexandrescu: On 03/16/2011 06:45 PM, bearophile wrote: - Where it is used it usually doesn't give a more meaningful exception like WrongArgumentException, etc. I don't want a deep hierarchy of one hundred standard exceptions, but I think some standard exceptions for the most common mistakes, like wrong arguments, etc, are better than a generic enforce(), especially for a standard library code that is meant to be written with care and to give better error messages/exceptions. enforce helps such idioms, does not prevent them. From the docs: === T enforce(T)(T value, lazy Throwable ex); If value is nonzero, returns it. Otherwise, throws ex. === Really? using enforce with a custom throwable saves *one* char: enforce(foo, new BlaException(bad!)); if(!foo) throw new BlaException(bad!); or are there other merits? enforce is an expression that returns its argument so it's composable. Andrei
Re: Against enforce()
Am 17.03.2011 02:55, schrieb Andrei Alexandrescu: On 03/16/2011 08:50 PM, Daniel Gibson wrote: Am 17.03.2011 02:07, schrieb Andrei Alexandrescu: On 03/16/2011 06:45 PM, bearophile wrote: - Where it is used it usually doesn't give a more meaningful exception like WrongArgumentException, etc. I don't want a deep hierarchy of one hundred standard exceptions, but I think some standard exceptions for the most common mistakes, like wrong arguments, etc, are better than a generic enforce(), especially for a standard library code that is meant to be written with care and to give better error messages/exceptions. enforce helps such idioms, does not prevent them. From the docs: === T enforce(T)(T value, lazy Throwable ex); If value is nonzero, returns it. Otherwise, throws ex. === Really? using enforce with a custom throwable saves *one* char: enforce(foo, new BlaException(bad!)); if(!foo) throw new BlaException(bad!); or are there other merits? enforce is an expression that returns its argument so it's composable. Andrei You're right. I just realized it and wanted to reply that correction to myself but you were faster :) I agree, that *is* helpful. A side note: The documentation says the value is returned if it's non-zero. enforce() also throws on null (for classes, probably also for pointers) and false (for bool..) - this should probably be mentioned in the docs ;-) Cheers, - Daniel
Re: Against enforce()
Daniel Gibson enforce(foo, new BlaException(bad!)); One use of enforce is something more like this: auto fp = enforce(fopen(my file, r), cant open file); Which replaces: auto fp = fopen(my file, r); if(fp is null) throw new Exception(cant open file);
Re: Against enforce()
On Wednesday, March 16, 2011 18:53:06 bearophile wrote: Andrei: Yum, love the enforce. You are silly :-) So perhaps the language could be improved as enforce does not break purity. Currently enforce() uses a lazy argument. A lazy argument is a delegate, and generally such delegate can't be pure, because the expressions you give to enforce() usually refer to variables in the scope where you call enforce(). So what kind of language improvements are you thinking about? This is an example: http://d.puremagic.com/issues/show_bug.cgi?id=5746 - It kills inlining (with the current DMD, and I don't think this problem will be fixed soon); Not a problem of enforce. In my opinion it's not wise use widely in the standard library something that requires an optimization that the DMD compiler is not going to have soon, and that makes the code significantly slower. In some cases this almost forces me to keep a patched version of Phobos, or to add more nearly duplicated functions to my dlibs2. enforce helps such idioms, does not prevent them. From the docs: === T enforce(T)(T value, lazy Throwable ex); If value is nonzero, returns it. Otherwise, throws ex. === Then why is iota() using a nude enforce() instead of the enforce() with a more meaningful exception like WrongArgumentException? I have seen the nude enforce used in other parts of Phobos. So maybe enforce() makes even the standard library writers lazy. enforce's specification specifies it throws. It would therefore be difficult for it to not throw. This complaint is non sequitur. I know, and I agree in some situations you want an assert (to test arguments coming from outside), so in some situations an enforce is acceptable. The problem is that currently enforce is used as a patch for a DMD/Phobos problem that I hope will be fixed in a short time, see below. I see enforce() just as a temporary workaround for a problem of Phobos (that it's compiled in release mode, so its asserts are vanished) that risks to become a permanent part of Phobos. enforce is a simple abstraction of the idiom if (!condition) throw new Exception(args). If that idiom were rare, then occurrences of enforce would be rare and therefore there would be little need to have enforce at all. There may be some confusion somewhere. enforce is not supposed to be a sort of assert. It is a different tool with a different charter. Use assert for assertions. Right. But saying just that is not enough. You have to add that such if (!condition) throw new Exception(args) idiom is common in Phobos because Phobos is present only in release mode. If the zip distribution of DMD contains two Phobos and dmd becomes able to use the right one according to the compilation switches, then I think that if (!condition) throw new Exception(args) will become more rare, and the enforce() too will be less commonly needed. For the most part, I don't think that this is true. There was discussion of using enforce instead of assertions because of assertions being compiled out with -release, but I'm not sure that that ever actually happened. Every case that I can think of at the moment where enforce is used, an exception _should_ be used. The only question is whether enforce should be used due to the current problems with inlining it and whatnot. - Jonathan M Davis
