Re: Null references redux
Lutger: > First, there is a huge demand for programmers, so much that even I got hired > in this time of crisis ;) Good programmers don't suddenly fall from the > skies apparently. This is the nicest thing I've read this week. Thank you very much :-) Biologists aren't that lucky, apparently. Bye, bearophile
Re: Null references redux
On 27-9-2009 9:20, Walter Bright wrote:
language_fan wrote:
The idea behind non-nullable types and other contracts is to catch
these errors on compile time. Sure, the code is a bit harder to write,
but it is safe and never segfaults. The idea is to minimize the amount
of runtime errors of all sorts. That's also how other features of
statically typed languages work.
I certainly agree that catching errors at compile time is preferable by
far. Where I disagree is the notion that non-nullable types achieve
this. I've argued extensively here that they hide errors, not fix them.
Also, by "safe" I presume you mean "memory safe" which means free of
memory corruption. Null pointer exceptions are memory safe. A null
pointer could be caused by memory corruption, but it cannot *cause*
memory corruption.
// t.d
void main()
{
int* a;
a[2] = 2;
}
[C:\Users\Lionello] dmd -run t.d
[C:\Users\Lionello]
This code passes on Vista. Granted, needs a big enough offset and some
luck, but indexing null will never be secure in the current flat memory
models.
L.
Re: Null references redux
grauzone, el 27 de septiembre a las 22:31 me escribiste: > >>Woah, nice. I stand corrected. Is this in druntime already? > > > >Not yet, its part of a custom runtime I'm working on and wish to > >release under a public domain license when I get the time. The > >code is linked from a thread in D.announce. > > Some of this functionality is also in Tango (SVN version). Signals > are catched only to print a backtrace. I think this is a very bad idea. When the program receive a segfault I want my lovely core dumped. A core dump is way more useful than any possible backtrace. I really don't see any use for it except if an uncaught exception could generate a core dump (just as GCC do for C++ code). But I *really* *really* want my core dump, so I can open my debugger and inspect the dead program exactly in the point where it failed. -- Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/ GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) The average person laughs 13 times a day
Re: Null references redux
Michel Fortin wrote:
On 2009-09-27 09:41:03 -0400, Andrei Alexandrescu
said:
Michel Fortin wrote:
On 2009-09-26 23:28:30 -0400, Michel Fortin
said:
On 2009-09-26 22:07:00 -0400, Walter Bright
said:
[...] The facilities in D enable one to construct a non-nullable
type, and they are appropriate for many designs. I just don't see
them as a replacement for *all* reference types.
As far as I understand this thread, no one here is arguing that
non-nullable references/pointers should replace *all*
reference/pointer types. The argument made is that non-nullable
should be the default and nullable can be specified explicitly any
time you need it.
So if you need a reference you use "Object" as the type, and if you
want that reference to be nullable you write "Object?". The static
analysis can then assert that your code properly check for null
prior dereferencing a nullable type and issues a compilation error
if not.
I just want to add: some people here are suggesting the compiler adds
code to check for null and throw exceptions... I believe like you
that this is the wrong approach because, like you said, it makes
people add dummy try/catch statements to ignore the error. What you
want a prorammer to do is check for null and properly handle the
situation before the error occurs, and this is exactly what the
static analysis approach I suggest forces.
Take this example where "a" is non-nullable and "b" is nullable:
string test(Object a, Object? b)
{
auto x = a.toString();
auto y = b.toString();
return x ~ y;
}
This should result in a compiler error on line 4 with a message
telling you that "b" needs to be checked for null prior use. The
programmer must then fix his error with an if (or some other control
structure), like this:
string test(Object a, Object? b)
{
audo result = a.toString();
if (b)
result ~= b.toString();
return result;
}
And now the compiler will let it pass. This is what I'd like to see.
What do you think?
I'm not totally against throwing exceptions in some cases, but the
above approach would be much more useful. Unfortunatly, throwing
exceptions it the best you can do with a library type approach.
I don't think this would fly.
You want me to add wings? Please explain.
I did explain. You suggest that we replace an automated, no-cost
checking with a manual, compulsory, conservative, and costly scheme.
That pretty much summarizes its disadvantages too :o).
Andrei
Re: The Non-Virtual Interface idiom in D
On Fri, 25 Sep 2009 16:49:27 -0400, Andrei Alexandrescu wrote: This leads to the interesting setup in which an interface should ideally define some signatures of to-be-defined functions, but disallow client code from calling them. For the clients, the same interface should expose some higher-level final functions. There are also some other benefits to allowing defining interface implementation, such as defining one function in terms of another (public) function. For example, take a look at: http://www.dsource.org/projects/dcollections/docs/current/dcollections.model.Collection.html Look at the definition for both remove functions, the one which takes no bool can easily be defined in terms of the one that takes a bool reference (in fact, this is what I do in all the implementations). The only issue is that you run into multiple inheritance issues (as brought up by Yigal)... vote++ -Steve
Re: Null references redux
BCS: > But this still assumes some degree of reliability of the code doing the fuzzy > logic. Fuzzy logic can also be "run" by hardware, fuzzy logic engine chips. Such chips are usually cheap. You can find them in some consumer products. The fuzzy logic rules can also be converted to correct programs by automatic software generators. (Of course mistakes etc are always possible.) Bye, bearophile
Re: Null references redux
Andrei Alexandrescu wrote: Jeremie Pelletier wrote: Is this Linux specific? what about other *nix systems, like BSD and solaris? Signal handler are standard to most *nix platforms since they're part of the posix C standard libraries, maybe some platforms will require a special handling but nothing impossible to do. Let me write a message on behalf of Sean Kelly. He wrote that to Walter and myself this morning, then I suggested him to post it but probably he is off email for a short while. Hopefully the community will find a solution to the issue he's raising. Let me post this: === Sean Kelly wrote: There's one minor problem with his code. It's not safe to throw an exception from a signal handler. Here's a quote from the POSIX spec at opengroup.org: "In order to prevent errors arising from interrupting non-reentrant function calls, applications should protect calls to these functions either by blocking the appropriate signals or through the use of some programmatic semaphore (see semget() , sem_init() , sem_open() , and so on). Note in particular that even the "safe" functions may modify errno; the signal-catching function, if not executing as an independent thread, may want to save and restore its value. Naturally, the same principles apply to the reentrancy of application routines and asynchronous data access. Note thatlongjmp() and siglongjmp() are not in the list of reentrant functions. This is because the code executing after longjmp() and siglongjmp() can call any unsafe functions with the same danger as calling those unsafe functions directly from the signal handler. Applications that use longjmp() andsiglongjmp() from within signal handlers require rigorous protection in order to be portable." If this were an acceptable approach it would have been in druntime ages ago :-) === Andrei Yes but the segfault signal handler is not made to design code that can live with these exceptions, its just a feature to allow segfaults to be sent to the crash handler to get a backtrace dump. Even on windows while you can recover from access violations, its generally a bad idea to allow for bugs to be turned into features. Jeremie
Re: The Non-Virtual Interface idiom in D
I think this is great, especially fine to see final methods in interfaces. Looks a bit like aspect oriented programming. Just wanted to add that it's not always *only* a benefit. Like with template method, flow of control may be a bit more complicated for the people who do the overriding. But that depends on how you use it though.
Re: Null references redux
Jarrett Billingsley: > And if you have a nullable reference that you know is not null for the > rest of the function? Just put "assert(x !is null)" and everything > that follows will assume it's not null. Asserts tend to vanish in release mode, so it may be better to use something different. A possibility is to use the enforce() some people have shown here. Another possibility is the very strange assume() of Visual C++, that I may appreciate for other purposes too: http://msdn.microsoft.com/en-us/library/1b3fsfxw(loband).aspx Bye, bearophile
Re: Null references redux
On 2009-09-27 09:41:03 -0400, Andrei Alexandrescu
said:
Michel Fortin wrote:
On 2009-09-26 23:28:30 -0400, Michel Fortin said:
On 2009-09-26 22:07:00 -0400, Walter Bright said:
[...] The facilities in D enable one to construct a non-nullable type,
and they are appropriate for many designs. I just don't see them as a
replacement for *all* reference types.
As far as I understand this thread, no one here is arguing that
non-nullable references/pointers should replace *all* reference/pointer
types. The argument made is that non-nullable should be the default and
nullable can be specified explicitly any time you need it.
So if you need a reference you use "Object" as the type, and if you
want that reference to be nullable you write "Object?". The static
analysis can then assert that your code properly check for null prior
dereferencing a nullable type and issues a compilation error if not.
I just want to add: some people here are suggesting the compiler adds
code to check for null and throw exceptions... I believe like you that
this is the wrong approach because, like you said, it makes people add
dummy try/catch statements to ignore the error. What you want a
prorammer to do is check for null and properly handle the situation
before the error occurs, and this is exactly what the static analysis
approach I suggest forces.
Take this example where "a" is non-nullable and "b" is nullable:
string test(Object a, Object? b)
{
auto x = a.toString();
auto y = b.toString();
return x ~ y;
}
This should result in a compiler error on line 4 with a message telling
you that "b" needs to be checked for null prior use. The programmer
must then fix his error with an if (or some other control structure),
like this:
string test(Object a, Object? b)
{
audo result = a.toString();
if (b)
result ~= b.toString();
return result;
}
And now the compiler will let it pass. This is what I'd like to see.
What do you think?
I'm not totally against throwing exceptions in some cases, but the
above approach would be much more useful. Unfortunatly, throwing
exceptions it the best you can do with a library type approach.
I don't think this would fly.
You want me to add wings? Please explain.
One good thing about nullable references is that they are dynamically
checked for validity at virtually zero cost.
When you say they are dynamically checked, do you mean it throws an
exception when you assign null? I'm not totally against this idea, but
I find the above a supperior solution because it forces the programmer
to handle the problem where it occurs and it doesn't require any
runtime check.
Non-nullable references, therefore, would not add value in that
respect, but would add value by reducing the cases when programmers
forgot to initialize references properly.
To me it looks like you're supporting an inferior concept for
non-nullable references because the better one will not "fly" (whatever
that means). Well, I support both concepts of non-nullable because they
both can be very useful, but I believe static checking is a better way
than throwing exceptions.
--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/
Re: Null references redux
Hello Lutger, BCS wrote: Hello Lutger, The answer may depend on [...] the habits of the 'programmers' in question, I don't know. If you can't trust the programmer to write good code, replace them with someone you can trust. There will never be a usable language that can take in garbage and spit out correct programs. Hi. I don't think this argument will work, for several reasons: [...] Do you maintain that a programmer who can't deal with non-nullable references without hacking them away is unusually incompetent? Incompetent? No. But I wouldn't want to hire a programer that *habitually* (and unnecessarily) hacks past a feature designed to prevent bugs. The best race car driver in the world is clearly not incompetent but would still get a ticket on public roads for speeding or following to close. I don't know about this. Actually I suspect non-nullable references by default are in the end safer (whatever that means), but only if they don't complicate the use of nullable references. I'll second that.
Re: The Non-Virtual Interface idiom in D
On 2009-09-27 09:53:12 -0400, Andrei Alexandrescu said: Then your examples should have shown this instead. Herb's article has them! From what I read, Herb's article is focussing on making classes ABI less fragile by removing virtual functions from public view and forcing users to call them behind a non-virtual functions. My take on this is that it's useful, but unintuitive. There is one example where he separates a task into multiple virtual functions called apparently in sequence from non-virtual one. That's a fine concept, but putting this into something called "interface" seems dubious to me as it looks more like an implementation detail than an interface. Still, having a way to write a default implementation for a function in an interface seems useful to me in some situations, but not in those illustrated the article you pointed to. I fully support having a way to specify a default implementation for a function in an interface. It might get handy for a few things (like implementing the delegate pattern you see everywhere in Cocoa). But it's a bad replacement for contracts. Walter has implemented contract inheritance, and we hope to be able to have contracts on interfaces in too. The former is a nice expected feature; the latter could convince DbC skeptics to start using it. That's really great! :-) -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: opEquals and the Non-Virtual Interface idiom
Andrei Alexandrescu wrote: Do you know of a precedent for this? One thing that NVI has going for it is that it's quite established - it seems to be solidly planted in programmers' lore. I was surprised to find 2.8M Google matches for the exact string "Non-Virtual Interface". Looks like it's considered a design pattern, and design patterns are popular. I think the idea itself is relatively obvious and useful, so it's likely to encounter it often. (I didn't even know it's called "NVI".) By the way, Aspect Oriented Programming also seems to reinvent this idea with pre/post code for methods (see Yigal's posting).
Re: Null references redux
Jeremie Pelletier wrote: Is this Linux specific? what about other *nix systems, like BSD and solaris? Signal handler are standard to most *nix platforms since they're part of the posix C standard libraries, maybe some platforms will require a special handling but nothing impossible to do. Let me write a message on behalf of Sean Kelly. He wrote that to Walter and myself this morning, then I suggested him to post it but probably he is off email for a short while. Hopefully the community will find a solution to the issue he's raising. Let me post this: === Sean Kelly wrote: There's one minor problem with his code. It's not safe to throw an exception from a signal handler. Here's a quote from the POSIX spec at opengroup.org: "In order to prevent errors arising from interrupting non-reentrant function calls, applications should protect calls to these functions either by blocking the appropriate signals or through the use of some programmatic semaphore (see semget() , sem_init() , sem_open() , and so on). Note in particular that even the "safe" functions may modify errno; the signal-catching function, if not executing as an independent thread, may want to save and restore its value. Naturally, the same principles apply to the reentrancy of application routines and asynchronous data access. Note thatlongjmp() and siglongjmp() are not in the list of reentrant functions. This is because the code executing after longjmp() and siglongjmp() can call any unsafe functions with the same danger as calling those unsafe functions directly from the signal handler. Applications that use longjmp() andsiglongjmp() from within signal handlers require rigorous protection in order to be portable." If this were an acceptable approach it would have been in druntime ages ago :-) === Andrei
Re: Null references redux
Jeremie Pelletier wrote: downs wrote: Jeremie Pelletier wrote: Andrei Alexandrescu wrote: downs wrote: Walter Bright wrote: Nick Sabalausky wrote: I agree with you that if the compiler can detect null dereferences at compile time, it should. Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption. No, he's using the real meaning of "safe", not the misleadingly-limited "SafeD" version of "safe" (which I'm still convinced is going to get some poor soul into serious trouble from mistakingly thinking their SafeD program is much safer than it really is). Out here in reality, "safe" also means a lack of ability to crash, or at least some level of protection against it. Memory safety is something that can be guaranteed (presuming the compiler is correctly implemented). There is no way to guarantee that a non-trivial program cannot crash. It's the old halting problem. Okay, I'm gonna have to call you out on this one because it's simply incorrect. The halting problem deals with a valid program state - halting. We cannot check if every program halts because halting is an instruction that must be allowed at almost any point in the program. Why do crashes have to be allowed? They're not an allowed instruction! A compiler can be turing complete and still not allow crashes. There is nothing wrong with this, and it has *nothing* to do with the halting problem. You seem to be under the impression that nothing can be made uncrashable without introducing the possibility of corrupted state. That's hogwash. I read that statement several times and I still don't understand what it means. BTW, hardware null pointer checking is a safety feature, just like array bounds checking is. PS: You can't convert segfaults into exceptions under Linux, as far as I know. How did Jeremie do that? Andrei A signal handler with the undocumented kernel parameters attaches the signal context to the exception object, repairs the stack frame forged by the kernel to make us believe we called the handler ourselves, does a backtrace right away and attaches it to the exception object, and then throw it. The error handling code will unwind down to the runtime's main() where a catch clause is waiting for any Throwables, sending them back into the unhandled exception handler, and a crash window appears with the backtrace, all finally blocks executed, and gracefully shutting down. All I need to do is an ELF/DWARF reader to extract symbolic debug info under linux, its already working for PE/CodeView on windows. Jeremie Woah, nice. I stand corrected. Is this in druntime already? Not yet, its part of a custom runtime I'm working on and wish to release under a public domain license when I get the time. The code is linked from a thread in D.announce. Some of this functionality is also in Tango (SVN version). Signals are catched only to print a backtrace.
Re: Null references redux
BCS wrote: > Hello Lutger, > >> The answer may >> depend on [...] >> the habits of the 'programmers' in question, I don't know. >> > > If you can't trust the programmer to write good code, replace them with > someone you can trust. There will never be a usable language that can take > in garbage and spit out correct programs. Hi. I don't think this argument will work, for several reasons: First, there is a huge demand for programmers, so much that even I got hired in this time of crisis ;) Good programmers don't suddenly fall from the skies apparently. Second, there are lot's of tasks doable by programmers with less skill than others using tools that trade safety for performance / expressiveness / whatever. Finally, programmers are humans, humans make mistakes, have quirks and bad days. All of them. What it comes down to is that languages are made in order to service and adapt to the programmers, not the other way around. Do you maintain that a programmer who can't deal with non-nullable references without hacking them away is unusually incompetent? I don't know about this. Actually I suspect non-nullable references by default are in the end safer (whatever that means), but only if they don't complicate the use of nullable references.
Re: Null references redux
downs wrote: Jeremie Pelletier wrote: Andrei Alexandrescu wrote: downs wrote: Walter Bright wrote: Nick Sabalausky wrote: I agree with you that if the compiler can detect null dereferences at compile time, it should. Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption. No, he's using the real meaning of "safe", not the misleadingly-limited "SafeD" version of "safe" (which I'm still convinced is going to get some poor soul into serious trouble from mistakingly thinking their SafeD program is much safer than it really is). Out here in reality, "safe" also means a lack of ability to crash, or at least some level of protection against it. Memory safety is something that can be guaranteed (presuming the compiler is correctly implemented). There is no way to guarantee that a non-trivial program cannot crash. It's the old halting problem. Okay, I'm gonna have to call you out on this one because it's simply incorrect. The halting problem deals with a valid program state - halting. We cannot check if every program halts because halting is an instruction that must be allowed at almost any point in the program. Why do crashes have to be allowed? They're not an allowed instruction! A compiler can be turing complete and still not allow crashes. There is nothing wrong with this, and it has *nothing* to do with the halting problem. You seem to be under the impression that nothing can be made uncrashable without introducing the possibility of corrupted state. That's hogwash. I read that statement several times and I still don't understand what it means. BTW, hardware null pointer checking is a safety feature, just like array bounds checking is. PS: You can't convert segfaults into exceptions under Linux, as far as I know. How did Jeremie do that? Andrei A signal handler with the undocumented kernel parameters attaches the signal context to the exception object, repairs the stack frame forged by the kernel to make us believe we called the handler ourselves, does a backtrace right away and attaches it to the exception object, and then throw it. The error handling code will unwind down to the runtime's main() where a catch clause is waiting for any Throwables, sending them back into the unhandled exception handler, and a crash window appears with the backtrace, all finally blocks executed, and gracefully shutting down. All I need to do is an ELF/DWARF reader to extract symbolic debug info under linux, its already working for PE/CodeView on windows. Jeremie Woah, nice. I stand corrected. Is this in druntime already? Not yet, its part of a custom runtime I'm working on and wish to release under a public domain license when I get the time. The code is linked from a thread in D.announce.
Re: opEquals and the Non-Virtual Interface idiom
Jeremie Pelletier wrote:
> grauzone wrote:
>> Andrei Alexandrescu wrote:
>>> Here's an article about the perils of equals in Java (opEquals in D):
>>>
>>> http://www.ddj.com/article/printableArticle.jhtml;jsessionid=GFKUCQH5S4IHNQE1GHOSKHWATMY32JVN?articleID=184405053&dept_url=/java/
>>>
>>>
>>> It turns out this is a great example for NVI. In D, we could and should
>>> do the following:
>>>
>>> class Object {
>>> // Implement this
>>> private bool opEqualsImpl(Object rhs) {
>>> return false;
>>> }
>>> // Use this
>>> final bool opEquals(Object rhs) {
>>> if (this is rhs) return true;
>>> if (this is null || rhs is null) return false;
>>> return opEqualsImpl(rhs) && rhs.opEqualsImpl(this);
>>> }
>>> }
>>>
>>> I took advantage of the fact that in a final function this may be null
>>> without an access violation. The implementation above ensures symmetry
>>> of equality and has each class implement a simpler primitive.
>>>
>>> What do you think?
>>
>> Eh, now after all this discussion, we're going to allow even "this" to
>> be null? That seems like a backstep...
>
> How is it a backstep? It's perfectly valid behavior.
>
> Object foo;
> foo.opEquals(foo);
>
What is the semantics of that call?
"Is something undefined equal to something undefined"? I'm not sure why that
should be considered valid behavior, given that it's unanswerable.
> The call itself will *always* succeed, its when 'this' gets referenced
> in opEquals that the code will crash.
Or when a contract is run. Or when an inherited contract is run. Or when the
method attempts to lock the synchronization object.
>
>> Implementing opEquals as a global/static function, that calls the
>> actual Object.opEquals virtual method would be so much more straight
>> forward.
>
> I agree, I prefer methods to end with Impl to stay hidden instead of
> being the ones to override.
>
>> PS: I agree about the NVI thing. If you'd go to extend the language
>> for "NVI", couldn't we just introduce a second type of virtual
>> function that works this way:
>> 1. the super class' implementation is _always_ called first
>> 2. the super class function can decide to "call down" to the sub
>> class' implementation of the same method
>> => no extra do method needed, and the code is (possibly)
>> clearer.
>>
>>> Andrei
>
> I don't know how useful that would be, when you override a method you
> usually want to call the super method somewhere in the new
> implementation, not always at the beginning.
Re: Null references redux
Jeremie Pelletier wrote:
> downs wrote:
>> Jeremie Pelletier wrote:
>>> Christopher Wright wrote:
Jeremie Pelletier wrote:
> What if using 'Object obj;' raises a warning "unitialized variable"
> and makes everyone wanting non-null references happy, and 'Object obj
> = null;' raises no warning and makes everyone wanting to keep the
> current system (all two of us!) happy.
>
> I believe it's a fair compromise.
It's a large improvement, but only for local variables. If your
segfault has to do with a local variable, unless your function is
monstrously large, it should be easy to fix, without changing the type
system.
The larger use case is when you have an aggregate member that cannot
be null. This can be solved via contracts, but they are tedious to
write and ubiquitous.
>>> But how would you enforce a nonnull type over an aggregate in the first
>>> place? If you can, you could also apply the same initializer semantics I
>>> suggested earlier.
>>>
>>> Look at this for example:
>>>
>>> struct A {
>>> Object cannotBeNull;
>>> }
>>>
>>> void main() {
>>> A* a = new A;
>>> }
>>>
>>> Memory gets initialized to zero, and you have a broken non-null type.
>>> You could have the compiler throw an error here, but the compiler cannot
>>> possibly know about all data creation methods such as malloc, calloc or
>>> any other external allocator.
>>>
>>> You could even do something like:
>>>
>>> Object* foo = calloc(Object.sizeof);
>>>
>>> and the compiler would let you dereference foo resulting in yet another
>>> broken nonnull variable.
>>>
>>> Non-nulls are a cute idea when you have a type system that is much
>>> stricter than D's, but there are just way too many workarounds to make
>>> it crash in D.
>>
>> "Here are some cases you haven't mentioned yet. This proves that the
>> compiler can't possibly be smart enough. "
>>
>> Yeah.
>
> I allocate most structs on the gc, unless I need them only for the scope
> of a function (that includes RVO). All objects are on the gc already, so
> it's a pretty major case. The argument was to protect aggregate fields,
> I'm just pointing out that their usage usually is preventing an easy
> implementation. I'm not saying its impossible.
>
> Besides, what I said was, if its possible to enforce these fields to be
> null/non-null, you can enforce them to be properly initialized in such
> case, making nulls/non-nulls nearly useless.
>
>> In the above case, why not implicitly put the cannotBeNull check into
>> the struct invariant? That's where it belongs, imho.
>
> Exactly, what's the need for null/non-null types then?
>
You're twisting my words.
Checking for null in the struct invariant would be an _implementation_ of
non-nullable types in structs.
Isn't the whole point of defaulting to non-nullable types that we don't have to
check for it manually, i.e. in the user-defined invariant?
I think we should avoid having to build recursive checks for null-ness for
every type we define.
>> Regarding your example, it's calloc(size_t.sizeof). And a) we probably
>> can't catch that case except with in/out null checks on every method,
>> but then again, how often have you done that? I don't think it's
>> relevant enough to be relevant to this thread. :)
>
> Actually, sizeof currently returns the size of the reference, so its
> always going to be the same as size_t.sizeof.
Weird. I remembered that differently. Thanks.
Re: Null references redux
Jeremie Pelletier wrote: > Andrei Alexandrescu wrote: >> downs wrote: >>> Walter Bright wrote: Nick Sabalausky wrote: I agree with you that if the compiler can detect null dereferences at compile time, it should. >> Also, by "safe" I presume you mean "memory safe" which means free of >> memory corruption. Null pointer exceptions are memory safe. A null >> pointer could be caused by memory corruption, but it cannot *cause* >> memory corruption. > No, he's using the real meaning of "safe", not the > misleadingly-limited "SafeD" version of "safe" (which I'm still > convinced is going to get some poor soul into serious trouble from > mistakingly thinking their SafeD program is much safer than it really > is). Out here in reality, "safe" also means a lack of ability to > crash, or at least some level of protection against it. Memory safety is something that can be guaranteed (presuming the compiler is correctly implemented). There is no way to guarantee that a non-trivial program cannot crash. It's the old halting problem. >>> >>> Okay, I'm gonna have to call you out on this one because it's simply >>> incorrect. >>> >>> The halting problem deals with a valid program state - halting. >>> >>> We cannot check if every program halts because halting is an >>> instruction that must be allowed at almost any point in the program. >>> >>> Why do crashes have to be allowed? They're not an allowed instruction! >>> >>> A compiler can be turing complete and still not allow crashes. There >>> is nothing wrong with this, and it has *nothing* to do with the >>> halting problem. >>> > You seem to be under the impression that nothing can be made > uncrashable without introducing the possibility of corrupted state. > That's hogwash. I read that statement several times and I still don't understand what it means. BTW, hardware null pointer checking is a safety feature, just like array bounds checking is. >>> >>> PS: You can't convert segfaults into exceptions under Linux, as far >>> as I know. >> >> How did Jeremie do that? >> >> Andrei > > A signal handler with the undocumented kernel parameters attaches the > signal context to the exception object, repairs the stack frame forged > by the kernel to make us believe we called the handler ourselves, does a > backtrace right away and attaches it to the exception object, and then > throw it. > > The error handling code will unwind down to the runtime's main() where a > catch clause is waiting for any Throwables, sending them back into the > unhandled exception handler, and a crash window appears with the > backtrace, all finally blocks executed, and gracefully shutting down. > > All I need to do is an ELF/DWARF reader to extract symbolic debug info > under linux, its already working for PE/CodeView on windows. > > Jeremie Woah, nice. I stand corrected. Is this in druntime already?
Re: Null references redux
Hello bearophile, Do you know fuzzy logic? One of the purposes of fuzzy logic is to design control systems (that can be used for washing machines, cameras, missiles, etc) that work and fail gracefully. They don't work in two binary ways perfect/totallywrong. A graceful failure may have avoided the Ariane to crash and go boom. Today people are studying software systems based on fuzzy logic, neural networks, support vector machines, and more, that are designed to keep working despite some small problems and faults. But this still assumes some degree of reliability of the code doing the fuzzy logic. If I had to guess, I'd expect that the systems you mention are designed to function under external faults (some expected input vanishes or some other component in a distributed system fails). It would be almost impossible to make a program that can work correctly once it has had an internal fault. Once that has happened, I think Walter is correct and the only thing to do is shut down. In the auto pilot case, this could amount to kill off the current auto pilot process and boot up a very simple fly-straight-and-level program to take over while the pilot reacts to a nice loud klaxon.
Re: Null references redux
On Sun, Sep 27, 2009 at 3:42 PM, Jeremie Pelletier wrote:
> Jarrett Billingsley wrote:
>> Nonnull types do not create implicit null checks. Nonnull types DO NOT
>> need to be checked. And nullable types WOULD force explicit null
>> checks.
>
> Forcing checks on nullables is just as bad, not all nullables need to be
> checked every time they're used.
You don't get it, do you. If you have a reference that doesn't need to
be checked every time it's used, you make it a *nonnull reference*.
You *only* use nullable variables for references where the nullness of
the reference should change the program logic.
And if you're talking about things like:
Foo? x = someFunc();
if(x is null)
{
// one path
}
else
{
// use x here
}
and you're expecting the "use x here" clause to force you to do
(cast(Foo)x) every time you want to use x? That's not the case. The
condition of the if has *proven* x to be nonnull in the else clause,
so no null checks - at compile time or at runtime - have to be
performed there, nor does it have to be cast to a nonnull reference.
And if you have a nullable reference that you know is not null for the
rest of the function? Just put "assert(x !is null)" and everything
that follows will assume it's not null.
>> hash_t foo(Object o) { return o.toHash(); }
>> foo(null); // bamf, I just killed your function.
>>
>> Forcing initialization of locals does NOT solve all the problems that
>> nonnull references would.
>
> You didn't kill my function, you shot yourself in the foot. Something
> trivial to debug.
You're dodging. You claim that forcing variable initialization solves
the same problem that nonnull references do. It doesn't.
Re: Null references redux
bearophile wrote: Jeremie Pelletier: The focus should be about implementing variable initialization checks to the compiler, since this solves the issue with any variable, not just references. The flow analysis can also be reused for many other optimizations. Are you willing to give your help to implement about 5-10% if this feature? :-) Bye, bearophile Sure, I would love to help implement flow analysis, I don't know enough of the current dmd semantic analysis internals yet, but I'm slowly getting there. Jeremie
Re: Null references redux
Hello Walter, Jeremie Pelletier wrote: It could even be used for any type, the default initializer in D is a cute idea, but not a performance friendly one. I would much prefer the compiler to allow "int a" but warn me if I use it before assigning anything to it than assigning it to zero, and then assigning it to the value I wanted. "= void" is nice but I'm pretty sure I'm way over a thousand uses of it so far. The compiler, when -O is used, should remove nearly all the redundant initializations. Sweet, so you already have a bunch of the logic needed to check make sure non-null references get initialized.
Re: Null references redux
Jarrett Billingsley wrote:
On Sun, Sep 27, 2009 at 2:07 PM, Jeremie Pelletier wrote:
Yes and no. It introduces an "if" statement for null checking, but only
for nullable references. If you know your reference can't be null it should
be non-nullable, and then you don't need to check.
I much prefer explicit null checks than implicit ones I can't control.
Nonnull types do not create implicit null checks. Nonnull types DO NOT
need to be checked. And nullable types WOULD force explicit null
checks.
Forcing checks on nullables is just as bad, not all nullables need to be
checked every time they're used.
What about non-nan floats? Or non-invalid characters? I fear nonnull
references are a first step in the wrong direction. The focus should be
about implementing variable initialization checks to the compiler, since
this solves the issue with any variable, not just references. The flow
analysis can also be reused for many other optimizations.
hash_t foo(Object o) { return o.toHash(); }
foo(null); // bamf, I just killed your function.
Forcing initialization of locals does NOT solve all the problems that
nonnull references would.
You didn't kill my function, you shot yourself in the foot. Something
trivial to debug.
Re: Null references redux
Hello Lutger, The answer may depend on [...] the habits of the 'programmers' in question, I don't know. If you can't trust the programmer to write good code, replace them with someone you can trust. There will never be a usable language that can take in garbage and spit out correct programs.
Re: Null references redux
Yigal Chripun wrote: On 27/09/2009 19:29, Jeremie Pelletier wrote: Andrei Alexandrescu wrote: downs wrote: Walter Bright wrote: Nick Sabalausky wrote: I agree with you that if the compiler can detect null dereferences at compile time, it should. Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption. No, he's using the real meaning of "safe", not the misleadingly-limited "SafeD" version of "safe" (which I'm still convinced is going to get some poor soul into serious trouble from mistakingly thinking their SafeD program is much safer than it really is). Out here in reality, "safe" also means a lack of ability to crash, or at least some level of protection against it. Memory safety is something that can be guaranteed (presuming the compiler is correctly implemented). There is no way to guarantee that a non-trivial program cannot crash. It's the old halting problem. Okay, I'm gonna have to call you out on this one because it's simply incorrect. The halting problem deals with a valid program state - halting. We cannot check if every program halts because halting is an instruction that must be allowed at almost any point in the program. Why do crashes have to be allowed? They're not an allowed instruction! A compiler can be turing complete and still not allow crashes. There is nothing wrong with this, and it has *nothing* to do with the halting problem. You seem to be under the impression that nothing can be made uncrashable without introducing the possibility of corrupted state. That's hogwash. I read that statement several times and I still don't understand what it means. BTW, hardware null pointer checking is a safety feature, just like array bounds checking is. PS: You can't convert segfaults into exceptions under Linux, as far as I know. How did Jeremie do that? Andrei A signal handler with the undocumented kernel parameters attaches the signal context to the exception object, repairs the stack frame forged by the kernel to make us believe we called the handler ourselves, does a backtrace right away and attaches it to the exception object, and then throw it. The error handling code will unwind down to the runtime's main() where a catch clause is waiting for any Throwables, sending them back into the unhandled exception handler, and a crash window appears with the backtrace, all finally blocks executed, and gracefully shutting down. All I need to do is an ELF/DWARF reader to extract symbolic debug info under linux, its already working for PE/CodeView on windows. Jeremie Is this Linux specific? what about other *nix systems, like BSD and solaris? Signal handler are standard to most *nix platforms since they're part of the posix C standard libraries, maybe some platforms will require a special handling but nothing impossible to do.
Re: Null references redux
Jeremie Pelletier: > The focus should be > about implementing variable initialization checks to the compiler, since > this solves the issue with any variable, not just references. The flow > analysis can also be reused for many other optimizations. Are you willing to give your help to implement about 5-10% if this feature? :-) Bye, bearophile
Re: Null references redux
Rainer Deyke:
> Of course, a good optimizer can still reorder the declarations in this
> case, or even eliminate the whole function body (since it doesn't do
> anything).
LLVM has a good optimizer. If you try the LLVM demo on C code with LTO
activated:
http://llvm.org/demo/index.cgi
This C code:
void bar(int foo) {
int a;
int c = 3;
if (foo) {
a = 1;
} else {
a = 2;
}
}
Produces an useful warining:
/tmp/webcompile/_16254_0.c:3: warning: unused variable 'c'
And an empty function:
define void @bar(i32 %foo) nounwind readnone {
entry:
ret void
}
Bye,
bearophile
Re: Null references redux
Jesse Phillips wrote:
> The thing is that memory safety is the only safety with code.
That is such bullshit. For example, this:
class A {
}
class B {
}
A x = new B;
No memory access violation (yet). Clearly incorrect. Detecting this at
compile time is clearly a safety feature, and a good one.
You could argue that assigned a 'B' to a variable that is declared to
hold an 'A' is already a memory safety violation. If so, then the exact
argument also applies to assigning 'null' to the same variable.
--
Rainer Deyke - rain...@eldwood.com
Re: Null references redux
Jeremie Pelletier wrote: > Walter Bright wrote: >> They are completely independent variables. One may get assigned to a >> register, and not the other. > > Ok, that's what I thought, so the good old C way of declaring variables > at the top is not a bad thing yet :) Strange how you can look at the evidence and arrive at exactly the wrong conclusion. Declaring variables as close as possible to where they are used can reduce stack usage, and never increases it. -- Rainer Deyke - rain...@eldwood.com
Re: Null references redux
Walter Bright wrote:
>> void bar(bool foo) {
>> int a = void;
>> if (foo) {
>> a = 1;
>> } else {
>> a = 2; // Reuse variable.
>> }
>> int c = 3;
>> }
>>
>> You now only have two variables, but both of them coexist at the end of
>> the function. Unless the compiler applies a clever optimization, the
>> compiler is now forced to allocate space for two variables on the stack.
>
> Not necessarily. The optimizer uses a technique called "live range
> analysis" to determine if two variables have non-overlapping ranges. It
> uses this for register assignment, but it could just as well be used for
> minimizing stack usage.
That's the optimization I was referring to. It works for ints, but not
for RAII types. It also doesn't (necessarily) work if you reorder the
function:
void bar(bool foo) {
int a = void;
int c = 3;
if (foo) {
a = 1;
} else {
a = 2; // Reuse variable.
}
}
Of course, a good optimizer can still reorder the declarations in this
case, or even eliminate the whole function body (since it doesn't do
anything).
--
Rainer Deyke - rain...@eldwood.com
Re: Null references redux
On 27/09/2009 19:29, Jeremie Pelletier wrote: Andrei Alexandrescu wrote: downs wrote: Walter Bright wrote: Nick Sabalausky wrote: I agree with you that if the compiler can detect null dereferences at compile time, it should. Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption. No, he's using the real meaning of "safe", not the misleadingly-limited "SafeD" version of "safe" (which I'm still convinced is going to get some poor soul into serious trouble from mistakingly thinking their SafeD program is much safer than it really is). Out here in reality, "safe" also means a lack of ability to crash, or at least some level of protection against it. Memory safety is something that can be guaranteed (presuming the compiler is correctly implemented). There is no way to guarantee that a non-trivial program cannot crash. It's the old halting problem. Okay, I'm gonna have to call you out on this one because it's simply incorrect. The halting problem deals with a valid program state - halting. We cannot check if every program halts because halting is an instruction that must be allowed at almost any point in the program. Why do crashes have to be allowed? They're not an allowed instruction! A compiler can be turing complete and still not allow crashes. There is nothing wrong with this, and it has *nothing* to do with the halting problem. You seem to be under the impression that nothing can be made uncrashable without introducing the possibility of corrupted state. That's hogwash. I read that statement several times and I still don't understand what it means. BTW, hardware null pointer checking is a safety feature, just like array bounds checking is. PS: You can't convert segfaults into exceptions under Linux, as far as I know. How did Jeremie do that? Andrei A signal handler with the undocumented kernel parameters attaches the signal context to the exception object, repairs the stack frame forged by the kernel to make us believe we called the handler ourselves, does a backtrace right away and attaches it to the exception object, and then throw it. The error handling code will unwind down to the runtime's main() where a catch clause is waiting for any Throwables, sending them back into the unhandled exception handler, and a crash window appears with the backtrace, all finally blocks executed, and gracefully shutting down. All I need to do is an ELF/DWARF reader to extract symbolic debug info under linux, its already working for PE/CodeView on windows. Jeremie Is this Linux specific? what about other *nix systems, like BSD and solaris?
Re: Null references redux
On Sun, Sep 27, 2009 at 2:07 PM, Jeremie Pelletier wrote:
>> Yes and no. It introduces an "if" statement for null checking, but only
>> for nullable references. If you know your reference can't be null it should
>> be non-nullable, and then you don't need to check.
>
> I much prefer explicit null checks than implicit ones I can't control.
Nonnull types do not create implicit null checks. Nonnull types DO NOT
need to be checked. And nullable types WOULD force explicit null
checks.
> What about non-nan floats? Or non-invalid characters? I fear nonnull
> references are a first step in the wrong direction. The focus should be
> about implementing variable initialization checks to the compiler, since
> this solves the issue with any variable, not just references. The flow
> analysis can also be reused for many other optimizations.
hash_t foo(Object o) { return o.toHash(); }
foo(null); // bamf, I just killed your function.
Forcing initialization of locals does NOT solve all the problems that
nonnull references would.
Re: opEquals and the Non-Virtual Interface idiom
On 27/09/2009 19:01, Andrei Alexandrescu wrote:
grauzone wrote:
Andrei Alexandrescu wrote:
Here's an article about the perils of equals in Java (opEquals in D):
http://www.ddj.com/article/printableArticle.jhtml;jsessionid=GFKUCQH5S4IHNQE1GHOSKHWATMY32JVN?articleID=184405053&dept_url=/java/
It turns out this is a great example for NVI. In D, we could and should
do the following:
class Object {
// Implement this
private bool opEqualsImpl(Object rhs) {
return false;
}
// Use this
final bool opEquals(Object rhs) {
if (this is rhs) return true;
if (this is null || rhs is null) return false;
return opEqualsImpl(rhs) && rhs.opEqualsImpl(this);
}
}
I took advantage of the fact that in a final function this may be null
without an access violation. The implementation above ensures symmetry
of equality and has each class implement a simpler primitive.
What do you think?
Eh, now after all this discussion, we're going to allow even "this" to
be null? That seems like a backstep...
Good point.
Implementing opEquals as a global/static function, that calls the
actual Object.opEquals virtual method would be so much more straight
forward.
It's also less safe because people could call the incomplete primitive
by hand. With NVI in place nobody outside object.d could ever call
opEqualsImpl.
PS: I agree about the NVI thing. If you'd go to extend the language
for "NVI", couldn't we just introduce a second type of virtual
function that works this way:
1. the super class' implementation is _always_ called first
2. the super class function can decide to "call down" to the sub
class' implementation of the same method
=> no extra do method needed, and the code is (possibly)
clearer.
Do you know of a precedent for this? One thing that NVI has going for it
is that it's quite established - it seems to be solidly planted in
programmers' lore. I was surprised to find 2.8M Google matches for the
exact string "Non-Virtual Interface".
Andrei
This is a smalltalk idea -
method:
^ self method
by sending a message to self without implementing it you make the method
abstract.
of course you can add pre/post code.
I also seen this with the keyword inner:
class Foo {
void bar() {
... setup
inner();
...tear down
}
}
Re: opEquals and the Non-Virtual Interface idiom
grauzone wrote:
Andrei Alexandrescu wrote:
Here's an article about the perils of equals in Java (opEquals in D):
http://www.ddj.com/article/printableArticle.jhtml;jsessionid=GFKUCQH5S4IHNQE1GHOSKHWATMY32JVN?articleID=184405053&dept_url=/java/
It turns out this is a great example for NVI. In D, we could and should
do the following:
class Object {
// Implement this
private bool opEqualsImpl(Object rhs) {
return false;
}
// Use this
final bool opEquals(Object rhs) {
if (this is rhs) return true;
if (this is null || rhs is null) return false;
return opEqualsImpl(rhs) && rhs.opEqualsImpl(this);
}
}
I took advantage of the fact that in a final function this may be null
without an access violation. The implementation above ensures symmetry
of equality and has each class implement a simpler primitive.
What do you think?
Eh, now after all this discussion, we're going to allow even "this" to
be null? That seems like a backstep...
How is it a backstep? It's perfectly valid behavior.
Object foo;
foo.opEquals(foo);
The call itself will *always* succeed, its when 'this' gets referenced
in opEquals that the code will crash.
Implementing opEquals as a global/static function, that calls the actual
Object.opEquals virtual method would be so much more straight forward.
I agree, I prefer methods to end with Impl to stay hidden instead of
being the ones to override.
PS: I agree about the NVI thing. If you'd go to extend the language for
"NVI", couldn't we just introduce a second type of virtual function that
works this way:
1. the super class' implementation is _always_ called first
2. the super class function can decide to "call down" to the sub class'
implementation of the same method
=> no extra do method needed, and the code is (possibly)
clearer.
Andrei
I don't know how useful that would be, when you override a method you
usually want to call the super method somewhere in the new
implementation, not always at the beginning.
Re: Null references redux
Rainer Deyke wrote:
OT, but declaring the variable at the top of the function increases
stack size.
Example with changed variable names:
void bar(bool foo) {
if (foo) {
int a = 1;
} else {
int b = 2;
}
int c = 3;
}
In this example, there are clearly three different (and differently
named) variables, but their lifetimes do not overlap. Only one variable
can exist at a time, therefore the compiler only needs to allocate space
for one variable. Now, if you move your declaration to the top:
void bar(bool foo) {
int a = void;
if (foo) {
a = 1;
} else {
a = 2; // Reuse variable.
}
int c = 3;
}
You now only have two variables, but both of them coexist at the end of
the function. Unless the compiler applies a clever optimization, the
compiler is now forced to allocate space for two variables on the stack.
Not necessarily. The optimizer uses a technique called "live range
analysis" to determine if two variables have non-overlapping ranges. It
uses this for register assignment, but it could just as well be used for
minimizing stack usage.
Re: Null references redux
Michel Fortin wrote: On 2009-09-27 07:38:59 -0400, Christopher Wright said: I dislike these forced checks. Let's say you're dealing with a compiler frontend. You have a semantic node that just went through some semantic pass and is guaranteed, by flow control and contracts, to have a certain property initialized that was not initialized prior to that point. The programmer knows the value isn't null. The compiler shouldn't force checks. At most, it should have automated checks that disappear with -release. If the programmer knows a value isn't null, why not put the value in a nullable-reference in the first place? It may not be nonnull for the entire lifetime of the reference. Also, it introduces more nesting. Yes and no. It introduces an "if" statement for null checking, but only for nullable references. If you know your reference can't be null it should be non-nullable, and then you don't need to check. I much prefer explicit null checks than implicit ones I can't control. Also, unless the compiler's flow analysis is great, it's a nuisance -- you can see that the error is bogus and have to insert extra checks. First you're right, if the feature is implemented it should be well implemented. Second, if in a few place you don't want an "if" clause, you can always cast your nullable reference to a non-nullable one, explicitly bypassing the safeties. If you write a cast, you are making a consious decision of not checking for null, which is much better than the current situation where it's very easy to forget to check for null. That's just adding useless verbosity to the language. It should be fine to provide a requireNotNull template and leave it at that. It's fine to have such a template. But it's not nearly as useful. It definitely is, the whole point is about reference initializations, not what they can or can't initialize to. What about non-nan floats? Or non-invalid characters? I fear nonnull references are a first step in the wrong direction. The focus should be about implementing variable initialization checks to the compiler, since this solves the issue with any variable, not just references. The flow analysis can also be reused for many other optimizations.
Re: Null references redux
language_fan wrote: Sun, 27 Sep 2009 12:35:23 -0400, Jeremie Pelletier thusly wrote: language_fan wrote: Sun, 27 Sep 2009 00:08:50 -0400, Jeremie Pelletier thusly wrote: Ary Borenszweig wrote: Just out of curiosity: have you ever programmed in Java or C#? Nope, never got interested in these to tell the truth. I only did C, C++, D and x86 assembly in systems programming, I have quite a background in PHP and JavaScript also. So you only know imperative procedural programming + some features of hybrid OOP languages that are not even proper OOP languages. This is what I know best, yeah. I did a lot of work in functional programming too, but not enough to add them to the above list. What is proper OOP anyways? It's a feature offered by the language, not a critical design that must obey to some strict standard rules. Be it class based or prototype based, supporting single or multiple inheritance, using abstract base classes or interfaces, having funny syntax for ctors and whatnot or using the class name or even 'this', its still OOP. If you wan't to call me on not knowing 15 languages like you do, I have to call you on not knowing the differences in OOP models. I must say I have not studied languages that much, only the concepts and theory - starting from formal definitions like operational or denotational semantics, and some more informal ones. I can professionally write code in only about half a dozen languages, but learning new ones is trivial if the task requires it. Generally the common thing for proper pure OOP languages is 'everything is an object' mentality. Because of this property there is no strict distinction between primitive non-OOP types and OOP types in pure OOP languages. In some languages e.g. number values are objects. In others there are no static members and even classes are objects, so called meta- objects. In some way you can see this purity even in UML. If we go into details, various OOP languages have major differences in their semantics. What I meant above is that I know a lot of developers who have a similar background as you do. It is really easy to use all of those languages without actually using the OOP features in them, at least properly (for instance PHP does not even have a real OOP system, it is a cheap rip-off of mainstream languages - just look at the scoping rules). I have seen Java code where the developer never constructs new objects and only uses static methods because he fears the heap allocation is expensive. Discussing OOP and language concepts is really hard if you lack the theoretical underpinning. It is sad to say this but the best source for this knowledge are academic CS books, but nowadays even wikipedia is starting to have good articles on the subject. I agree, Wikipedia is often the first source I check to learn on different concepts, then I search for online papers and documentation, dig into source code (Google's code search is a gem), and finally books. I'm not most programmers, and I'm sure you aren't either. I like to learn as much of the semantics and implementation details behind a language as I can, only then do I feel I know the language, I like to make the best out of everything in the languages I use, not specialize in a subset of it. I don't believe in a perfect programming model, I believe in many different models each having their pros and cons that can live in the same language forming an all-around solution. That's why I usually stay away from 'pure' languages because they impose a single point of view of the world, that doesn't mean its a bad one, I just like to look at the world from different angles at the same time.
Re: Null references redux
Walter Bright: > I've never seen any suggestion that Boeing (or Airbus, or the FAA) has > changed its philosophy on this. Do you have a reference? I like to read many books, I have read about this in the chapter Cofee cups in the piloting room, in the book "Turn Signals Are The Facial Expressions Of Automobiles" by Donand Normand. It talks about the "strong silent type" of computer automation, discussed in "In the age of smart machine: The future of work and Power", by Zuboff. An example of such problem is explained in: NTSB 1986 Aircraft Accident report - China Airlines 747-SP, N4522V, 300 nautical miles northwest San Francisco, California, February 19, 1985 (Rapp. NTSB/AAR-86/03), Washington DC.: http://libraryonline.erau.edu/online-full-text/ntsb/aircraft-accident-reports/AAR86-03.pdf It shows a problem that a better design in the autopilot interface can avoid (probably things have improved since 1985). > I should also point out that this strategy has been extremely > successful. Flying is inherently dangerous, yet is statistically > incredibly safe. Boeing is doing a LOT right, and I would be extremely > cautious of changing the philosophy that so far has delivered > spectacular results. They keep improving all the time. > BTW, shutting off the autopilot does not cause the airplane to suddenly > nosedive. Airliner aerodynamics are designed to be stable and to seek > straight and level flight if the controls are not touched. Autopilots do > shut themselves off now and then, and the pilot takes command. See the example I've shown that shows why what you have said can be dangerous anyway. A sudden automatic switch off of the autopilot can be dangerous, because people need time to understand what's happening, when the situation goes from a mostly automatic one to a mostly manual one. > Please give an example. I'll give one. How about that crash in the > Netherlands recently where the autopilot decided to fly the airplane > into the ground? As I recall it was getting bad data from the > altimeters. I have a firm conviction that if there's a fault in the > altimeters, the pilot should be informed and get control back > immediately, as opposed to thinking about a sandwich (or whatever) while > the autopilot soldiered on. An emergency can escalate very, very fast > when you're going 600 mph. Of course the pilot must have a button to immediately regain manual control when she/he wants so. My point was different, that a sudden automatic full disable of autopilot can be dangerous. > Failing gracefully is done by shutting down the failed system and > engaging a backup, not by trying to convince yourself that a program in > an unknown state is capable of continuing to function. Software simply > does not work that way - one bit wrong and anything can happen. Software used to work that way in the past, but this is not set in stone. The famous crash of Ariane was caused by ultra-rigid reaction to errors in software. Do you know fuzzy logic? One of the purposes of fuzzy logic is to design control systems (that can be used for washing machines, cameras, missiles, etc) that work and fail gracefully. They don't work in two binary ways perfect/totallywrong. A graceful failure may have avoided the Ariane to crash and go boom. Today people are studying software systems based on fuzzy logic, neural networks, support vector machines, and more, that are designed to keep working despite some small problems and faults. In some situations (like a TAC machine in an hospital) you may want it to switch off totally if a problem is found, instead of a graceful failure. On the other hand if you put such TAC in a poor hospital in Africa you may want something that keeps working even if some small trouble is present, because a less than perfect machine is going to be the standard situation where there is very little money, and a reduced functionality TAC may save lot of people anyway (if it emits too much X rays it's better to switch it off). Bye, bearophile
Re: Null references redux
Sun, 27 Sep 2009 12:35:23 -0400, Jeremie Pelletier thusly wrote: > language_fan wrote: >> Sun, 27 Sep 2009 00:08:50 -0400, Jeremie Pelletier thusly wrote: >> >>> Ary Borenszweig wrote: Just out of curiosity: have you ever programmed in Java or C#? >>> Nope, never got interested in these to tell the truth. I only did C, >>> C++, D and x86 assembly in systems programming, I have quite a >>> background in PHP and JavaScript also. >> >> So you only know imperative procedural programming + some features of >> hybrid OOP languages that are not even proper OOP languages. > > This is what I know best, yeah. I did a lot of work in functional > programming too, but not enough to add them to the above list. > > What is proper OOP anyways? It's a feature offered by the language, not > a critical design that must obey to some strict standard rules. Be it > class based or prototype based, supporting single or multiple > inheritance, using abstract base classes or interfaces, having funny > syntax for ctors and whatnot or using the class name or even 'this', its > still OOP. If you wan't to call me on not knowing 15 languages like you > do, I have to call you on not knowing the differences in OOP models. I must say I have not studied languages that much, only the concepts and theory - starting from formal definitions like operational or denotational semantics, and some more informal ones. I can professionally write code in only about half a dozen languages, but learning new ones is trivial if the task requires it. Generally the common thing for proper pure OOP languages is 'everything is an object' mentality. Because of this property there is no strict distinction between primitive non-OOP types and OOP types in pure OOP languages. In some languages e.g. number values are objects. In others there are no static members and even classes are objects, so called meta- objects. In some way you can see this purity even in UML. If we go into details, various OOP languages have major differences in their semantics. What I meant above is that I know a lot of developers who have a similar background as you do. It is really easy to use all of those languages without actually using the OOP features in them, at least properly (for instance PHP does not even have a real OOP system, it is a cheap rip-off of mainstream languages - just look at the scoping rules). I have seen Java code where the developer never constructs new objects and only uses static methods because he fears the heap allocation is expensive. Discussing OOP and language concepts is really hard if you lack the theoretical underpinning. It is sad to say this but the best source for this knowledge are academic CS books, but nowadays even wikipedia is starting to have good articles on the subject.
Re: Null references redux
Jesse Phillips: >The thing is that memory safety is the only safety with code.< Nope. For example in Delphi and C# you can have a runtime integer overflow errors. That's another kind of safety. If you look at safety-critical code, the one Walter was talking about, you see people test code (and compile time) very well, looking for an enormous amount of possible errors. Doing this increases code safety. So you can have ABS brakes, TAC machine in hospitals, automatic pilots and so on. Bye, bearophile
Re: Null references redux
Andrei Alexandrescu wrote: downs wrote: Walter Bright wrote: Nick Sabalausky wrote: I agree with you that if the compiler can detect null dereferences at compile time, it should. Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption. No, he's using the real meaning of "safe", not the misleadingly-limited "SafeD" version of "safe" (which I'm still convinced is going to get some poor soul into serious trouble from mistakingly thinking their SafeD program is much safer than it really is). Out here in reality, "safe" also means a lack of ability to crash, or at least some level of protection against it. Memory safety is something that can be guaranteed (presuming the compiler is correctly implemented). There is no way to guarantee that a non-trivial program cannot crash. It's the old halting problem. Okay, I'm gonna have to call you out on this one because it's simply incorrect. The halting problem deals with a valid program state - halting. We cannot check if every program halts because halting is an instruction that must be allowed at almost any point in the program. Why do crashes have to be allowed? They're not an allowed instruction! A compiler can be turing complete and still not allow crashes. There is nothing wrong with this, and it has *nothing* to do with the halting problem. You seem to be under the impression that nothing can be made uncrashable without introducing the possibility of corrupted state. That's hogwash. I read that statement several times and I still don't understand what it means. BTW, hardware null pointer checking is a safety feature, just like array bounds checking is. PS: You can't convert segfaults into exceptions under Linux, as far as I know. How did Jeremie do that? Andrei A signal handler with the undocumented kernel parameters attaches the signal context to the exception object, repairs the stack frame forged by the kernel to make us believe we called the handler ourselves, does a backtrace right away and attaches it to the exception object, and then throw it. The error handling code will unwind down to the runtime's main() where a catch clause is waiting for any Throwables, sending them back into the unhandled exception handler, and a crash window appears with the backtrace, all finally blocks executed, and gracefully shutting down. All I need to do is an ELF/DWARF reader to extract symbolic debug info under linux, its already working for PE/CodeView on windows. Jeremie
Re: Null references redux
Hello Walter, The only reasonable thing a program can do if it discovers it is in an unknown state is to stop immediately. This whole thread is NOT about what to do on unknown states. It is about using the compiler to statically remove the possibility of one type of unknown state ever happening. If D were to get non-null by default, with optional nullable, then without ASM/union hacks or the like, you can only get a seg-v when you use the non-default nullable type. Given the above (and assuming memory safety), the only possible wrong-data-error left would be where the programmer explicitly places the wrong value in a variable. In my book, that is a non-starter because 1) it can happen now 2) it can happen anywhere, not just at initialization 3) it can't be detected and 4) (assuming a well done syntax) in the cases where the compiler can't validate the code, the lazy thing to do and the correct thing to do (use a nullable type) will be the same.
Re: opEquals and the Non-Virtual Interface idiom
grauzone wrote:
Andrei Alexandrescu wrote:
Here's an article about the perils of equals in Java (opEquals in D):
http://www.ddj.com/article/printableArticle.jhtml;jsessionid=GFKUCQH5S4IHNQE1GHOSKHWATMY32JVN?articleID=184405053&dept_url=/java/
It turns out this is a great example for NVI. In D, we could and should
do the following:
class Object {
// Implement this
private bool opEqualsImpl(Object rhs) {
return false;
}
// Use this
final bool opEquals(Object rhs) {
if (this is rhs) return true;
if (this is null || rhs is null) return false;
return opEqualsImpl(rhs) && rhs.opEqualsImpl(this);
}
}
I took advantage of the fact that in a final function this may be null
without an access violation. The implementation above ensures symmetry
of equality and has each class implement a simpler primitive.
What do you think?
Eh, now after all this discussion, we're going to allow even "this" to
be null? That seems like a backstep...
Good point.
Implementing opEquals as a global/static function, that calls the actual
Object.opEquals virtual method would be so much more straight forward.
It's also less safe because people could call the incomplete primitive
by hand. With NVI in place nobody outside object.d could ever call
opEqualsImpl.
PS: I agree about the NVI thing. If you'd go to extend the language for
"NVI", couldn't we just introduce a second type of virtual function that
works this way:
1. the super class' implementation is _always_ called first
2. the super class function can decide to "call down" to the sub class'
implementation of the same method
=> no extra do method needed, and the code is (possibly)
clearer.
Do you know of a precedent for this? One thing that NVI has going for it
is that it's quite established - it seems to be solidly planted in
programmers' lore. I was surprised to find 2.8M Google matches for the
exact string "Non-Virtual Interface".
Andrei
Re: Null references redux
On Sun, 27 Sep 2009 11:51:27 -0400, bearophile
wrote:
Steven Schveighoffer:
Build the non-null requirement into the function signature (note, the
requirement is optional, it's still possible to use null references if
you
want).
Pros: Easy to implement, Compile-time error, hard to "work around" by
putting a dummy value, sometimes no performance hit, most times very
little performance hit, allows solution 1 and 2 if you want, runtime
errors occur AT THE POINT things went wrong not later.
Cons: Non-zero performance hit (you have to check for null sometimes
before assignment!)
To implement it well (and I think it has to be implemented well) it's
not so easy to implement. You have to face the problem I've discussed
about about multiple object initializations inside various ifs.
I think you are referring to a combination of this solution and flow
analysis? I didn't mention that solution, but it is possible. I agree it
would be more complicated, but I did say that as a con for flow analysis.
Also see what downs and I have said regarding arrays of nonnullables.
Yes, arrays of non-nullables will be more cumbersome, I should add that as
a con. Thanks.
Among the cons you also have to consider that there's a little more
complexity in the language (two different kinds of references, and such
things must also be explained in the docs and understood by novice D
programmers. It's not a common feature, so they have to learn it).
It's not a common feature, but in practice, one doesn't usually need
nullable types for most cases, it's only certain cases where it's needed.
For example, no extra docs are needed for:
auto a = new A(); // works, non-nullable is fine
And maybe even for:
A a; // error, must assign non-null value
because that's a common feature of compilers.
It's similar in my view to shared. Shared adds a level of complexity that
needs to be understood if you want to use shared variables, but most of
the time, your variables are not shared, so no extra thought is required.
Another thing to add to the cons is that every layer of compile-time
constraints you add to a language they also add a little amount of
rigidity that has a cost (because you have to add ? and you sometimes
may need casts to break such rigidity). Dynamic languages show that
constraints have a cost.
The cost needs to be weighed against the cost of the alternatives. I
think all the solutions have a cost. Dynamic languages have a cost too.
I've been developing in php lately, and I don't know how many times I had
a bug that I slightly mis-typed a variable name, which still was valid
code because the language thought I was just declaring a new variable :)
And to get the IDE to recognize types, I sometimes have to put in a line
like this:
// uncomment for autocomplete
// x = new ClassType(); printf("Error, please remove line %d\n",
__LINE__); throw new Exception();
Which I comment out when I'm running, but I uncomment to have the IDE
recognize that x is a ClassType (for autocomplete).
I think if there was a solution that cost nothing, it would be the clear
winner.
-Steve
Re: Null references redux
Sun, 27 Sep 2009 16:47:51 +, Jesse Phillips thusly wrote: > The thing is that memory safety is the only safety with code. In > Walter's examples he very clearly showed that a crash is not unsafe, but > operating with incorrect values is. He has pointed out that if > initialization is enforced, whether with a default or by coder, there is > a good chance it will be initialized to the wrong value. Have you ever used functional languages? When you develop in Haskell or SML, how often you feel there is a good change something will be initialized to the wrong value? Can you show some statistics that show how unsafe this practice is? When the non-nullability is made optional, you *only* use it when you really know the initialization has a sane value, ok? Otherwise you can use the good old nullable references, right? > Now if you really want to throw some sticks into the spokes, you would > say that if the program crashes due to a null pointer, it is still > likely that the programmer will just initialize/set the value to a > "default" that still isn't valid just to get the program to continue to > run. Why should it crash in the first place? I hate crashes. You liek them? I can prove by structural induction that you do not like them when you can avoid crashes with static checking.
Re: opEquals and the Non-Virtual Interface idiom
Andrei Alexandrescu wrote:
Here's an article about the perils of equals in Java (opEquals in D):
http://www.ddj.com/article/printableArticle.jhtml;jsessionid=GFKUCQH5S4IHNQE1GHOSKHWATMY32JVN?articleID=184405053&dept_url=/java/
It turns out this is a great example for NVI. In D, we could and should
do the following:
class Object {
// Implement this
private bool opEqualsImpl(Object rhs) {
return false;
}
// Use this
final bool opEquals(Object rhs) {
if (this is rhs) return true;
if (this is null || rhs is null) return false;
return opEqualsImpl(rhs) && rhs.opEqualsImpl(this);
}
}
I took advantage of the fact that in a final function this may be null
without an access violation. The implementation above ensures symmetry
of equality and has each class implement a simpler primitive.
What do you think?
Eh, now after all this discussion, we're going to allow even "this" to
be null? That seems like a backstep...
Implementing opEquals as a global/static function, that calls the actual
Object.opEquals virtual method would be so much more straight forward.
PS: I agree about the NVI thing. If you'd go to extend the language for
"NVI", couldn't we just introduce a second type of virtual function that
works this way:
1. the super class' implementation is _always_ called first
2. the super class function can decide to "call down" to the sub class'
implementation of the same method
=> no extra do method needed, and the code is (possibly) clearer.
Andrei
Re: The Non-Virtual Interface idiom in D
Walter Bright wrote:
Andrei Alexandrescu wrote:
This feature would require changing some protection rules, I think for
the better. What do you think?
1. what are those changes?
I thought some more about it and here are the changes in protection rules:
1. Public final methods CANNOT be hijacked in derived classes by any
means. That means, if I see:
class Root {
final void fun(double) {
...
}
}
I can be darn sure that if I call obj.fun(5) against any class
inheriting Root, the call will go straight to Root.fun.
One nice thing about that is that we already have very solid
anti-hijacking rules in place, so we have a great precedence for
consistency.
2. Private non-final (overridable) methods must be overridable in
derived classes, even those defined in distinct modules.
3. A class overriding a method is NOT allowed to decrease encapsulation
by relaxing the protection level of an overridden method. In particular,
if a method is private and overridable, the function can override it but
is unable to call super.method.
I think the above rules are enough to implement a mean NVI.
2. did you mean to add executable code to an interface?
Yes. We need to allow final methods in interfaces. I remember I posted a
bug to that effect. These are not problematic to implement because have
the same binary regime as regular functions:
interface A {
void transmogrify();
final void transmogrifyTwice() {
transmogrify();
transmogrify();
}
}
I am very excited about this development, particularly because we can
get tricky things like opEquals right.
Andrei
Re: contravariant argument types: wanna?
BCS wrote:
Hello Walter,
Andrei Alexandrescu wrote:
Second, the new rule is simple: if the overriding function can be
called with the overriden function's arguments, it is overriding it.
True, things get more complicated when the base class also defines a
corresponding overload:
class A {
void fun(A);
void fun(B);
}
class B : A {
override void fun(A);
}
This must be either an error, or the case that B.fun overrides both
overloads of fun in A.
I would really want to get away from the notion of selecting which
function is overridden based on being a "better" match.
a torture test:
class B
{
void Fn (D,B) {}
void Fn (B,D) {}
}
class D : B
{
void Fn(B,B) {} // override D,B or B,D?
}
This is simple - it overrides both. There are more complicated cases
with longer inheritance chains and overloads defined in the derived
classes.
I think it's ok to hold off contravariance for now.
Andrei
Re: Null references redux
On Sun, 27 Sep 2009 10:10:19 -0400, Nick Sabalausky wrote: > "Walter Bright" wrote in message > news:h9n3k5$2eu...@digitalmars.com... >> Jason House wrote: Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption. >>> >>> I reject this argument too :( To me, code isn't safe if it crashes. >> >> Well, we can't discuss this if we cannot agree on terms. The >> conventional definition of memory safe means no memory corruption. > > He keeps saying "safe", and every time he does you turn it into "memory > safe". If he meant "memory safe" he probably would have said something > like "memory safe". He already made it perfectly clear he's talking > about crashes, so continuing to put the words "memory safe" into his > mouth doesn't help the discussion. The thing is that memory safety is the only safety with code. In Walter's examples he very clearly showed that a crash is not unsafe, but operating with incorrect values is. He has pointed out that if initialization is enforced, whether with a default or by coder, there is a good chance it will be initialized to the wrong value. Now if you really want to throw some sticks into the spokes, you would say that if the program crashes due to a null pointer, it is still likely that the programmer will just initialize/set the value to a "default" that still isn't valid just to get the program to continue to run.
Re: Null references redux
Hello downs, PS: You can't convert segfaults into exceptions under Linux, as far as I know. Last I checked, throwing from a signal handler works on linux.
Re: contravariant argument types: wanna?
Hello Walter,
Andrei Alexandrescu wrote:
Second, the new rule is simple: if the overriding function can be
called with the overriden function's arguments, it is overriding it.
True, things get more complicated when the base class also defines a
corresponding overload:
class A {
void fun(A);
void fun(B);
}
class B : A {
override void fun(A);
}
This must be either an error, or the case that B.fun overrides both
overloads of fun in A.
I would really want to get away from the notion of selecting which
function is overridden based on being a "better" match.
a torture test:
class B
{
void Fn (D,B) {}
void Fn (B,D) {}
}
class D : B
{
void Fn(B,B) {} // override D,B or B,D?
}
Re: The Non-Virtual Interface idiom in D
Jérôme M. Berger wrote: Michel Fortin wrote: I fully support having a way to specify a default implementation for a function in an interface. It might get handy for a few things (like implementing the delegate pattern you see everywhere in Cocoa). But it's a bad replacement for contracts. Then what's the difference between an interface and an abstract class? I thought that the whole point of interfaces was that you couldn't have implementations of the methods so that you had no problem choosing an implementation when inheriting from multiple interfaces. Jerome The interface supports multiple inheritance since it doesn't add to the vtable of the class using it, and its code would be implemented on the classes implementing the interface, not overridden by subclasses.
Re: Null references redux
language_fan wrote: Sun, 27 Sep 2009 00:08:50 -0400, Jeremie Pelletier thusly wrote: Ary Borenszweig wrote: Just out of curiosity: have you ever programmed in Java or C#? Nope, never got interested in these to tell the truth. I only did C, C++, D and x86 assembly in systems programming, I have quite a background in PHP and JavaScript also. So you only know imperative procedural programming + some features of hybrid OOP languages that are not even proper OOP languages. This is what I know best, yeah. I did a lot of work in functional programming too, but not enough to add them to the above list. What is proper OOP anyways? It's a feature offered by the language, not a critical design that must obey to some strict standard rules. Be it class based or prototype based, supporting single or multiple inheritance, using abstract base classes or interfaces, having funny syntax for ctors and whatnot or using the class name or even 'this', its still OOP. If you wan't to call me on not knowing 15 languages like you do, I have to call you on not knowing the differences in OOP models. I played with a lot of languages, but those are the ones I use on a daily basis. I would like to get into Python or Ruby someday, I only hear good things about these two. I know LUA has less overhead than Python Oh, the only difference between LUA and Python is the overhead?! That's a... pretty performance oriented view on languages. Yes, I have a performance oriented view, I write a lot of real time code, and I hate unresponsive code in general. Now I didn't say it was the only difference, what I said is that it's one influencing a lot companies and people to pick LUA over Python for scripting. I like extremes :) If you like extremes, why have you not programming in Haskell or Coq? Too scary? You are often arguing against languages and concepts you have never used. The other people here who make these suggestions are more experienced with various languages. I meant extremes as in full machine control / no control whatsoever, not in language semantics :) I just haven't found a use for Haskell or Coq for what I do yet.
opEquals and the Non-Virtual Interface idiom
Here's an article about the perils of equals in Java (opEquals in D):
http://www.ddj.com/article/printableArticle.jhtml;jsessionid=GFKUCQH5S4IHNQE1GHOSKHWATMY32JVN?articleID=184405053&dept_url=/java/
It turns out this is a great example for NVI. In D, we could and should
do the following:
class Object {
// Implement this
private bool opEqualsImpl(Object rhs) {
return false;
}
// Use this
final bool opEquals(Object rhs) {
if (this is rhs) return true;
if (this is null || rhs is null) return false;
return opEqualsImpl(rhs) && rhs.opEqualsImpl(this);
}
}
I took advantage of the fact that in a final function this may be null
without an access violation. The implementation above ensures symmetry
of equality and has each class implement a simpler primitive.
What do you think?
Andrei
Re: Null references redux
On 27/09/2009 17:51, bearophile wrote: Steven Schveighoffer: Build the non-null requirement into the function signature (note, the requirement is optional, it's still possible to use null references if you want). Pros: Easy to implement, Compile-time error, hard to "work around" by putting a dummy value, sometimes no performance hit, most times very little performance hit, allows solution 1 and 2 if you want, runtime errors occur AT THE POINT things went wrong not later. Cons: Non-zero performance hit (you have to check for null sometimes before assignment!) To implement it well (and I think it has to be implemented well) it's not so easy to implement. You have to face the problem I've discussed about about multiple object initializations inside various ifs. Also see what downs and I have said regarding arrays of nonnullables. I don't accept this argument about nested if statements. D has a procedural "if" statement. Of course it doesn't mesh together with non-nullable references, you're trying to fit a square peg in a round hole. the solution is to write a more functional style code. if D ever implements true tuples that would be a perfect use case for them. (T1 t1, T2 t2) = init(); t1.foo; t2.bar; Among the cons you also have to consider that there's a little more complexity in the language (two different kinds of references, and such things must also be explained in the docs and understood by novice D programmers. It's not a common feature, so they have to learn it). that's true. Not only this needs to be taught and pointed out to newbies it should also be encouraged as the D way so that it will be used by default. Another thing to add to the cons is that every layer of compile-time constraints you add to a language they also add a little amount of rigidity that has a cost (because you have to add ? and you sometimes may need casts to break such rigidity). Dynamic languages show that constraints have a cost. Bye, bearophile
opEquals and the Non-Virtual Interface idiom
Here's an article about the perils of equals in Java (opEquals in D):
http://www.ddj.com/article/printableArticle.jhtml;jsessionid=GFKUCQH5S4IHNQE1GHOSKHWATMY32JVN?articleID=184405053&dept_url=/java/
It turns out this is a great example for NVI. In D, we could and should
do the following:
class Object {
// Implement this
private bool opEqualsImpl(Object rhs);
// Use this
final bool opEquals(Object rhs) {
if (this is rhs) return true;
if (this is null || rhs is null) return false;
return opEqualsImpl(rhs) && rhs.opEqualsImpl(this);
}
}
I took advantage of the fact that in a final function this may be null
without an access violation. The implementation above ensures symmetry
of equality and has each class implement a simpler primitive.
What do you think?
Andrei
Re: Null references redux
downs wrote:
Jeremie Pelletier wrote:
Christopher Wright wrote:
Jeremie Pelletier wrote:
What if using 'Object obj;' raises a warning "unitialized variable"
and makes everyone wanting non-null references happy, and 'Object obj
= null;' raises no warning and makes everyone wanting to keep the
current system (all two of us!) happy.
I believe it's a fair compromise.
It's a large improvement, but only for local variables. If your
segfault has to do with a local variable, unless your function is
monstrously large, it should be easy to fix, without changing the type
system.
The larger use case is when you have an aggregate member that cannot
be null. This can be solved via contracts, but they are tedious to
write and ubiquitous.
But how would you enforce a nonnull type over an aggregate in the first
place? If you can, you could also apply the same initializer semantics I
suggested earlier.
Look at this for example:
struct A {
Object cannotBeNull;
}
void main() {
A* a = new A;
}
Memory gets initialized to zero, and you have a broken non-null type.
You could have the compiler throw an error here, but the compiler cannot
possibly know about all data creation methods such as malloc, calloc or
any other external allocator.
You could even do something like:
Object* foo = calloc(Object.sizeof);
and the compiler would let you dereference foo resulting in yet another
broken nonnull variable.
Non-nulls are a cute idea when you have a type system that is much
stricter than D's, but there are just way too many workarounds to make
it crash in D.
"Here are some cases you haven't mentioned yet. This proves that the compiler can't
possibly be smart enough. "
Yeah.
I allocate most structs on the gc, unless I need them only for the scope
of a function (that includes RVO). All objects are on the gc already, so
it's a pretty major case. The argument was to protect aggregate fields,
I'm just pointing out that their usage usually is preventing an easy
implementation. I'm not saying its impossible.
Besides, what I said was, if its possible to enforce these fields to be
null/non-null, you can enforce them to be properly initialized in such
case, making nulls/non-nulls nearly useless.
In the above case, why not implicitly put the cannotBeNull check into the
struct invariant? That's where it belongs, imho.
Exactly, what's the need for null/non-null types then?
Regarding your example, it's calloc(size_t.sizeof). And a) we probably can't
catch that case except with in/out null checks on every method, but then again,
how often have you done that? I don't think it's relevant enough to be relevant
to this thread. :)
Actually, sizeof currently returns the size of the reference, so its
always going to be the same as size_t.sizeof.
Re: Null references redux
Walter Bright wrote:
Jeremie Pelletier wrote:
This may be a good time to ask about how these variables which can be
declared anywhere in the function scope are implemented.
void bar(bool foo) {
if(foo) {
int a = 1;
...
}
else {
int a = 2;
...
}
}
is the stack frame using two ints, or is the compiler seeing only one?
I never bothered to check it out and just declared 'int a = void;' at
the beginning of the routine to keep the stack frames as small as
possible.
They are completely independent variables. One may get assigned to a
register, and not the other.
Ok, that's what I thought, so the good old C way of declaring variables
at the top is not a bad thing yet :)
Re: putting more smarts into a == b
Andrei Alexandrescu wrote:
Frank Benoit wrote:
Andrei Alexandrescu schrieb:
Frank Benoit wrote:
Andrei Alexandrescu schrieb:
Consider two objects a and b with a of class type. Currently, the
expression a == b is blindly rewritten as a.opEquals(b). I argue it
should be rewritten into a call to an (imaginary/inlined) function
equalObjects(a, b), with the following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also deals
with null references. What do you think?
Andrei
What about interfaces?
Good question! What do they do now? I ran this:
interface A {}
class Widget : A {}
void main() {
auto a = cast(A) new Widget;
A b = null;
writeln(a == b);
writeln(b == a);
}
To my surprise, the program printed false twice. If I replace A with
Widget inside main, the program prints false then crashes with the
mythical segfault :o).
So how are interfaces compared?
Andrei
Hm, i would have expected it not to compile, because A does not have
opEquals.
In DWT, I cast always first to Object.
Java> if( intf1.equals(intf2) ){
D1.0> if( ((cast(Object)intf1).opEquals( cast(Object)intf2 )){
I think in D the cast is inserted automatically. Walter?
Andrei
Using the compile-time view of Descent, if I have this code:
---
interface I {
}
class C {
}
int main(char[][] args) {
C c = new C();
I i = null;
auto x = i == c;
auto y = c == i;
return 0;
}
---
the compiler turns it into:
---
interface I {
}
class C: Object {
}
int main(char[][] args) {
C c = new C;
I i = null;
int x = c.opEquals(cast(Object) i);
int y = c.opEquals(cast(Object) i);
return 0;
}
---
That's why it doesn't segfault.
Debugging the code it turns out the logic is very simple, it's just
applying operator overloading: (for the first comparison) "opEquals" is
searched in "I". Since "opEquals" is not found in it, the compiler
checks if "opEquals" is commutative. It is, so it searches "opEquals" in
"C", finds it and does the rewrite. (more or less)
Re: Null references redux
Steven Schveighoffer: >Build the non-null requirement into the function signature (note, the > requirement is optional, it's still possible to use null references if you > want). > >Pros: Easy to implement, Compile-time error, hard to "work around" by > putting a dummy value, sometimes no performance hit, most times very > little performance hit, allows solution 1 and 2 if you want, runtime > errors occur AT THE POINT things went wrong not later. >Cons: Non-zero performance hit (you have to check for null sometimes > before assignment!) To implement it well (and I think it has to be implemented well) it's not so easy to implement. You have to face the problem I've discussed about about multiple object initializations inside various ifs. Also see what downs and I have said regarding arrays of nonnullables. Among the cons you also have to consider that there's a little more complexity in the language (two different kinds of references, and such things must also be explained in the docs and understood by novice D programmers. It's not a common feature, so they have to learn it). Another thing to add to the cons is that every layer of compile-time constraints you add to a language they also add a little amount of rigidity that has a cost (because you have to add ? and you sometimes may need casts to break such rigidity). Dynamic languages show that constraints have a cost. Bye, bearophile
Re: Null references redux
Lutger Wrote: > This is what it's about, I think: are non-nullable references *by default* > so annoying as to cause programmers to initialize them with wrong values (or > circumventing them in other ways)? > The answer may depend on the details of the feature, quality of > implementation and on the habits of the 'programmers' in question, I don't > know. In reality, the issue becomes what will programmers do to bypass compiler errors. This is one area where syntactic sugar is worth its weight in gold. I'm envisioning the syntax of Fan, or a very C#-like syntax: SomeType x; // Not nullable SomeType? y; // Nullable If the developer is too lazy to add the question mark and prefers to do SomeType x = cast(SomeType) null; Then it's their own fault when they get a runtime segfault to replace a compile-time error.
Re: Null references redux
On Sat, 26 Sep 2009 17:08:32 -0400, Walter Bright wrote: Denis Koroskin wrote: > On Sat, 26 Sep 2009 22:30:58 +0400, Walter Bright > wrote: >> D has borrowed ideas from many different languages. The trick is to >> take the good stuff and avoid their mistakes . > > How about this one: > http://sadekdrobi.com/2008/12/22/null-references-the-billion-dollar-mistake/ > > > :) I think he's wrong. Analogies aside, we have 2 distinct problems here, with several solutions for each. I jotted down what I think are the solutions being discussed and the Pros and Cons of each are. Problem 1. Developer of a function wants to ensure non-null values are passed into his function. Solution 1: Rely on the hardware feature to do the checking for you. Pros: Easy to do, simple to implement, optimal performance (hardware's going to do this anyways). Cons: Runtime error instead of compile-time, Error doesn't always occur close to the problem, not always easy to get a stack trace. Solution 2: Check for null once the values come into the function, throw an exception. Pros: Works with the exception system. Cons: Manual implementation required, performance hit for every function call, Runtime error instead of compile-time, Error doesn't always occur close to the problem. Solution 3: Build the non-null requirement into the function signature (note, the requirement is optional, it's still possible to use null references if you want). Pros: Easy to implement, Compile-time error, hard to "work around" by putting a dummy value, sometimes no performance hit, most times very little performance hit, allows solution 1 and 2 if you want, runtime errors occur AT THE POINT things went wrong not later. Cons: Non-zero performance hit (you have to check for null sometimes before assignment!) Solution 4: Perform a null check for every dereference (The Java/C# solution). Pros: Works with the exception system, easy to implement. Cons: Huge performance hit (except in OS where segfault can be hooked), Error doesn't always occur close to the problem. --- Problem 2. Developer forgets to initialize a declared reference type, but uses it. Solution 1: Assign a default value of null. Rely on hardware to tell you when you use it later that you screwed up. Pros: Easy to do, simple to implement, optimal performance (hardware's going to do this anyways). Cons: Runtime error instead of compile-time, Error doesn't always occur close to the problem, not always easy to get a stack trace. Solution 2: Require assignment, even if assignment to null. (The "simple" solution) Pros: Easy to implement, forces the developer to clarify his requirements -- reminding him that there may be a problem. Cons: May be unnecessary, forces the developer to make a decision, may result in a dummy value being assigned reducing to solution 1. Solution 3: Build into the type the requirement that it can't be null, therefore checking for non-null on assignment. A default value isn't allowed. A nullable type is still allowed, which reduces to solution 1. Pros: Easy to implement, solution 1 is still possible, compile-time error on misuse, error occurs at the point things went wrong, no performance hit (except when you convert a nullable type to a non-nullable type), allows solution 3 for first problem. Cons: Non-zero performance hit when assigning nullable to non nullable type. Solution 4: Compiler performs flow analysis, giving an error when an unassigned variable is used. (The C# solution) Pros: Compile-time error, with good flow analysis allows correct code even when assignment isn't done on declaration. Cons: Difficult to implement, sometimes can incorrectly require assignment if flow is too complex, can force developer to manually assign null or dummy value. *NOTE* for solution 3 I purposely did NOT include the con that it makes people assign a dummy value. I believe this argument to be invalid, since it's much easier to just declare the variable as a nullable equivalent type (as other people have pointed out). That problem is more a factor of solutions 2 and 4. -- Anything I missed? After looking at all the arguments, and brainstorming myself, I think I prefer the non-nullable defaults (I didn't have a position on this concept before this thread, and I had given it some thought). I completely agree with Ary and some others who say "use C# for a while, and see how much it helps." I wrote C# code for a while, and I got those errors frequently, usually it was something I forgot to initialize or return. It definitely does not cause the "assign dummy value" syndrome as Walter has suggested. Experience with languages that do a good job of letting the programmer know when he made an actual mistake makes a huge difference. I think the non-null
Re: putting more smarts into a == b
Robert Jacques wrote:
On Sun, 27 Sep 2009 09:11:38 -0400, Andrei Alexandrescu
wrote:
Robert Jacques wrote:
On Sat, 26 Sep 2009 21:32:13 -0400, Andrei Alexandrescu
wrote:
Consider two objects a and b with a of class type. Currently, the
expression a == b is blindly rewritten as a.opEquals(b). I argue it
should be rewritten into a call to an (imaginary/inlined) function
equalObjects(a, b), with the following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also
deals with null references. What do you think?
Andrei
I like this. I think optimizing away opEquals for identical objects
would also be a good idea:
static if (is(U == class))
if(a is b || a is null || b is null) return a is b;
else
enforce(a !is null);
This code has an inefficiency, it seems, because it makes a bit more
checks than necessary (e.g. checks a is b twice). Let's simplify:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (a is b) return true;
if (b is null || a is null) return false;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
Andrei
Are the extra branch and return statement faster?
Just as fast. Short-circuit evaluation also generates code with branches.
Besides, I thought the
optimizer would cache a is b:
auto a_is_b = a is b;
if (a_is_b || b is null || a is null) return a_is_b;
I'm trying to not rely on such...
Andrei
Re: putting more smarts into a == b
On Sun, 27 Sep 2009 09:11:38 -0400, Andrei Alexandrescu
wrote:
Robert Jacques wrote:
On Sat, 26 Sep 2009 21:32:13 -0400, Andrei Alexandrescu
wrote:
Consider two objects a and b with a of class type. Currently, the
expression a == b is blindly rewritten as a.opEquals(b). I argue it
should be rewritten into a call to an (imaginary/inlined) function
equalObjects(a, b), with the following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also deals
with null references. What do you think?
Andrei
I like this. I think optimizing away opEquals for identical objects
would also be a good idea:
static if (is(U == class))
if(a is b || a is null || b is null) return a is b;
else
enforce(a !is null);
This code has an inefficiency, it seems, because it makes a bit more
checks than necessary (e.g. checks a is b twice). Let's simplify:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (a is b) return true;
if (b is null || a is null) return false;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
Andrei
Are the extra branch and return statement faster? Besides, I thought the
optimizer would cache a is b:
auto a_is_b = a is b;
if (a_is_b || b is null || a is null) return a_is_b;
Re: The Non-Virtual Interface idiom in D
Michel Fortin wrote: I fully support having a way to specify a default implementation for a function in an interface. It might get handy for a few things (like implementing the delegate pattern you see everywhere in Cocoa). But it's a bad replacement for contracts. Then what's the difference between an interface and an abstract class? I thought that the whole point of interfaces was that you couldn't have implementations of the methods so that you had no problem choosing an implementation when inheriting from multiple interfaces. Jerome -- mailto:jeber...@free.fr http://jeberger.free.fr Jabber: jeber...@jabber.fr signature.asc Description: OpenPGP digital signature
Re: putting more smarts into a == b
On Sun, 27 Sep 2009 10:32:29 -0400, Andrei Alexandrescu
wrote:
Frank Benoit wrote:
Andrei Alexandrescu schrieb:
Frank Benoit wrote:
Andrei Alexandrescu schrieb:
Consider two objects a and b with a of class type. Currently, the
expression a == b is blindly rewritten as a.opEquals(b). I argue it
should be rewritten into a call to an (imaginary/inlined) function
equalObjects(a, b), with the following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also
deals
with null references. What do you think?
Andrei
What about interfaces?
Good question! What do they do now? I ran this:
interface A {}
class Widget : A {}
void main() {
auto a = cast(A) new Widget;
A b = null;
writeln(a == b);
writeln(b == a);
}
To my surprise, the program printed false twice. If I replace A with
Widget inside main, the program prints false then crashes with the
mythical segfault :o).
So how are interfaces compared?
Andrei
Hm, i would have expected it not to compile, because A does not have
opEquals.
In DWT, I cast always first to Object.
Java> if( intf1.equals(intf2) ){
D1.0> if( ((cast(Object)intf1).opEquals( cast(Object)intf2 )){
I think in D the cast is inserted automatically. Walter?
From the assembly, it appears that the compiler is comparing the reference
values directly. Which is not what you want, you want the opEquals from
Object.
-Steve
Re: putting more smarts into a == b
Andrei Alexandrescu schrieb:
> Frank Benoit wrote:
>> In DWT, I cast always first to Object.
>> Java> if( intf1.equals(intf2) ){
>> D1.0> if( ((cast(Object)intf1).opEquals( cast(Object)intf2 )){
>
> I think in D the cast is inserted automatically. Walter?
>
> Andrei
there is a related bug report
http://d.puremagic.com/issues/show_bug.cgi?id=2794
Re: Null references redux
Nick Sabalausky wrote: > "Walter Bright" wrote in message ... > You're acting as if handling failures safely and minimizing failures were > mutually exclusive. Not that I have an opinion on this either way, but if I understand Walter right that is exactly his point (although you exaggerate it a bit), see below. >> It's not designed to segfault. It's designed to expose errors, not hide >> them. > > Right. And some of these errors can be exposed at compile time...and you > want to just leave them as runtime segfaults instead? And you want this > because exposing an error at compile time somehow causes it to become a > hidden error? somehow -> encourages a practice where programmers get annoyed by the 'exposing of errors' to the point that they hide them This is what it's about, I think: are non-nullable references *by default* so annoying as to cause programmers to initialize them with wrong values (or circumventing them in other ways)? The answer may depend on the details of the feature, quality of implementation and on the habits of the 'programmers' in question, I don't know.
Re: Null references redux
"Walter Bright" wrote in message news:h9n44k$2g6...@digitalmars.com... > > Memory safety is something that can be guaranteed (presuming the compiler > is correctly implemented). There is no way to guarantee that a non-trivial > program cannot crash. It's the old halting problem. > No one said anything about guaranteeing a lack of *any* crash at all. But *some* crashes *can* be guaranteed not to occur. Specifically, a function that takes in a non-nullable reference 'bar' can be guaranteed not to exhibit a null dereference crash upon a dereference of 'bar'. And that guarantee can be made without making the code any more suseptable to corrupted state that it would otherwise be. > BTW, hardware null pointer checking is a safety feature, just like array > bounds checking is. Sure. And a runtime check to make sure a multiplication is between two numbers is also a safety feature...But we have a better way to handle that.
Re: putting more smarts into a == b
Frank Benoit wrote:
Andrei Alexandrescu schrieb:
Frank Benoit wrote:
Andrei Alexandrescu schrieb:
Consider two objects a and b with a of class type. Currently, the
expression a == b is blindly rewritten as a.opEquals(b). I argue it
should be rewritten into a call to an (imaginary/inlined) function
equalObjects(a, b), with the following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also deals
with null references. What do you think?
Andrei
What about interfaces?
Good question! What do they do now? I ran this:
interface A {}
class Widget : A {}
void main() {
auto a = cast(A) new Widget;
A b = null;
writeln(a == b);
writeln(b == a);
}
To my surprise, the program printed false twice. If I replace A with
Widget inside main, the program prints false then crashes with the
mythical segfault :o).
So how are interfaces compared?
Andrei
Hm, i would have expected it not to compile, because A does not have
opEquals.
In DWT, I cast always first to Object.
Java> if( intf1.equals(intf2) ){
D1.0> if( ((cast(Object)intf1).opEquals( cast(Object)intf2 )){
I think in D the cast is inserted automatically. Walter?
Andrei
Re: Null references redux
Nick Sabalausky: > He keeps saying "safe", and every time he does you turn it into "memory > safe". If he meant "memory safe" he probably would have said something like > "memory safe". He already made it perfectly clear he's talking about > crashes, so continuing to put the words "memory safe" into his mouth doesn't > help the discussion. Likewise, I think that the name of SafeD modules is misleading, they are MemorySafeD :-) Bye, bearophile
Re: putting more smarts into a == b
Andrei Alexandrescu schrieb:
> Frank Benoit wrote:
>> Andrei Alexandrescu schrieb:
>>> Consider two objects a and b with a of class type. Currently, the
>>> expression a == b is blindly rewritten as a.opEquals(b). I argue it
>>> should be rewritten into a call to an (imaginary/inlined) function
>>> equalObjects(a, b), with the following definition:
>>>
>>> bool equalObjects(T, U)(T a, U b) if (is(T == class))
>>> {
>>> static if (is(U == class))
>>> {
>>> if (b is null) return a is null;
>>> if (a is null) return b is null;
>>> }
>>> else
>>> {
>>> enforce(a !is null);
>>> }
>>> return a.opEquals(b);
>>> }
>>>
>>> This hoists the identity test outside the opEquals call and also deals
>>> with null references. What do you think?
>>>
>>>
>>> Andrei
>>
>> What about interfaces?
>
> Good question! What do they do now? I ran this:
>
> interface A {}
> class Widget : A {}
>
> void main() {
> auto a = cast(A) new Widget;
> A b = null;
> writeln(a == b);
> writeln(b == a);
> }
>
> To my surprise, the program printed false twice. If I replace A with
> Widget inside main, the program prints false then crashes with the
> mythical segfault :o).
>
> So how are interfaces compared?
>
>
> Andrei
Hm, i would have expected it not to compile, because A does not have
opEquals.
In DWT, I cast always first to Object.
Java> if( intf1.equals(intf2) ){
D1.0> if( ((cast(Object)intf1).opEquals( cast(Object)intf2 )){
Re: Null references redux
"Walter Bright" wrote in message news:h9n3k5$2eu...@digitalmars.com... > Jason House wrote: >>> Also, by "safe" I presume you mean "memory safe" which means free >>> of memory corruption. Null pointer exceptions are memory safe. A >>> null pointer could be caused by memory corruption, but it cannot >>> *cause* memory corruption. >> >> I reject this argument too :( To me, code isn't safe if it crashes. > > Well, we can't discuss this if we cannot agree on terms. The conventional > definition of memory safe means no memory corruption. He keeps saying "safe", and every time he does you turn it into "memory safe". If he meant "memory safe" he probably would have said something like "memory safe". He already made it perfectly clear he's talking about crashes, so continuing to put the words "memory safe" into his mouth doesn't help the discussion. > Boeing, Boeing, Boeing, Boeing, Boeing... Straw man. No one's arguing against designing systems to survive failure, and no one's arguing against forcing errors to be exposed. Your point seems to be: A good system is designed to handle a crash/failure without corruption, so let's allow things to crash/fail all they want. Our point is: A good system is designed to handle a crash/failure without corruption, but let's also do what we can to minimize the amount of crashes/failures in the first place. You're acting as if handling failures safely and minimizing failures were mutually exclusive. > It's not designed to segfault. It's designed to expose errors, not hide > them. Right. And some of these errors can be exposed at compile time...and you want to just leave them as runtime segfaults instead? And you want this because exposing an error at compile time somehow causes it to become a hidden error?
Re: Null references redux
Andrei Alexandrescu: > One good thing about nullable references > is that they are dynamically checked for validity at virtually zero > cost. Non-nullable references, therefore, would not add value in that > respect, but would add value by reducing the cases when programmers > forgot to initialize references properly. nonnullable references can also reduce the total amount of code a little, because you don't need to write the null tests often (the points where you use objects are more than the points where you instantiate them). Bye, bearophile
Re: The Non-Virtual Interface idiom in D
Michel Fortin wrote: On 2009-09-26 10:06:24 -0400, Andrei Alexandrescu said: Michel Fortin wrote: I think you're writing a lot of boilerplate code for something that the compiler should be able to do by itself. I mean, it's a lot cleaner with contracts, and there is no reason the compiler couldn't generate itself that "contract-verifying" non-virtual function. I think it would be a mistake to latch on my quick examples. It's not only about before and after checks, it's more about low-level customization points versus higher-level interfaces. Then your examples should have shown this instead. Herb's article has them! I fully support having a way to specify a default implementation for a function in an interface. It might get handy for a few things (like implementing the delegate pattern you see everywhere in Cocoa). But it's a bad replacement for contracts. Walter has implemented contract inheritance, and we hope to be able to have contracts on interfaces in too. The former is a nice expected feature; the latter could convince DbC skeptics to start using it. Andrei
Re: Null references redux
downs wrote: Walter Bright wrote: Nick Sabalausky wrote: I agree with you that if the compiler can detect null dereferences at compile time, it should. Also, by "safe" I presume you mean "memory safe" which means free of memory corruption. Null pointer exceptions are memory safe. A null pointer could be caused by memory corruption, but it cannot *cause* memory corruption. No, he's using the real meaning of "safe", not the misleadingly-limited "SafeD" version of "safe" (which I'm still convinced is going to get some poor soul into serious trouble from mistakingly thinking their SafeD program is much safer than it really is). Out here in reality, "safe" also means a lack of ability to crash, or at least some level of protection against it. Memory safety is something that can be guaranteed (presuming the compiler is correctly implemented). There is no way to guarantee that a non-trivial program cannot crash. It's the old halting problem. Okay, I'm gonna have to call you out on this one because it's simply incorrect. The halting problem deals with a valid program state - halting. We cannot check if every program halts because halting is an instruction that must be allowed at almost any point in the program. Why do crashes have to be allowed? They're not an allowed instruction! A compiler can be turing complete and still not allow crashes. There is nothing wrong with this, and it has *nothing* to do with the halting problem. You seem to be under the impression that nothing can be made uncrashable without introducing the possibility of corrupted state. That's hogwash. I read that statement several times and I still don't understand what it means. BTW, hardware null pointer checking is a safety feature, just like array bounds checking is. PS: You can't convert segfaults into exceptions under Linux, as far as I know. How did Jeremie do that? Andrei
Re: putting more smarts into a == b
Frank Benoit wrote:
Andrei Alexandrescu schrieb:
Consider two objects a and b with a of class type. Currently, the
expression a == b is blindly rewritten as a.opEquals(b). I argue it
should be rewritten into a call to an (imaginary/inlined) function
equalObjects(a, b), with the following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also deals
with null references. What do you think?
Andrei
What about interfaces?
Good question! What do they do now? I ran this:
interface A {}
class Widget : A {}
void main() {
auto a = cast(A) new Widget;
A b = null;
writeln(a == b);
writeln(b == a);
}
To my surprise, the program printed false twice. If I replace A with
Widget inside main, the program prints false then crashes with the
mythical segfault :o).
So how are interfaces compared?
Andrei
Re: Null references redux
Michel Fortin wrote:
On 2009-09-26 23:28:30 -0400, Michel Fortin
said:
On 2009-09-26 22:07:00 -0400, Walter Bright
said:
[...] The facilities in D enable one to construct a non-nullable
type, and they are appropriate for many designs. I just don't see
them as a replacement for *all* reference types.
As far as I understand this thread, no one here is arguing that
non-nullable references/pointers should replace *all*
reference/pointer types. The argument made is that non-nullable should
be the default and nullable can be specified explicitly any time you
need it.
So if you need a reference you use "Object" as the type, and if you
want that reference to be nullable you write "Object?". The static
analysis can then assert that your code properly check for null prior
dereferencing a nullable type and issues a compilation error if not.
I just want to add: some people here are suggesting the compiler adds
code to check for null and throw exceptions... I believe like you that
this is the wrong approach because, like you said, it makes people add
dummy try/catch statements to ignore the error. What you want a
prorammer to do is check for null and properly handle the situation
before the error occurs, and this is exactly what the static analysis
approach I suggest forces.
Take this example where "a" is non-nullable and "b" is nullable:
string test(Object a, Object? b)
{
auto x = a.toString();
auto y = b.toString();
return x ~ y;
}
This should result in a compiler error on line 4 with a message telling
you that "b" needs to be checked for null prior use. The programmer must
then fix his error with an if (or some other control structure), like this:
string test(Object a, Object? b)
{
audo result = a.toString();
if (b)
result ~= b.toString();
return result;
}
And now the compiler will let it pass. This is what I'd like to see.
What do you think?
I'm not totally against throwing exceptions in some cases, but the above
approach would be much more useful. Unfortunatly, throwing exceptions it
the best you can do with a library type approach.
I don't think this would fly. One good thing about nullable references
is that they are dynamically checked for validity at virtually zero
cost. Non-nullable references, therefore, would not add value in that
respect, but would add value by reducing the cases when programmers
forgot to initialize references properly.
Andrei
Re: Null references redux
Walter Bright wrote:
Andrei Alexandrescu wrote:
Walter Bright wrote:
Justin Johansson wrote:
Walter, in the heat of this thread I hope you haven't missed the
correlation with discussion
on "Dispatching on a variant" and noting:
Thanks for pointing it out. The facilities in D enable one to
construct a non-nullable type, and they are appropriate for many
designs.
No. There is no means to disable default construction.
Ack, I remember we talked about this, I guess I don't remember the
resolution.
The resolution was that the language will allow delete'ing the unwanted
constructor:
struct NonNull(T) if (is(T == class))
{
delete this();
...
}
Andrei
Re: Null references redux
Jeremie Pelletier wrote: Walter Bright wrote: Yigal Chripun wrote: An exception trace is *far* better than a segfault and that does not require null values. Seg faults are exceptions, too. You can even catch them (on windows)! Walter, check the crash handler I submitted to D.announce, it has signal handlers on linux to convert segfaults into D exception objects and throw them so the code can unwind properly and even catch it. It has made my life so much easier, I barely need to run within a debugger anymore for most crashes. I don't know enough of phobos and druntime to port it, but its under a public domain license so anyone is free to do it! I think that's great. Walter, Sean, please let's look into this. Andrei
Re: putting more smarts into a == b
Robert Jacques wrote:
On Sat, 26 Sep 2009 21:32:13 -0400, Andrei Alexandrescu
wrote:
Consider two objects a and b with a of class type. Currently, the
expression a == b is blindly rewritten as a.opEquals(b). I argue it
should be rewritten into a call to an (imaginary/inlined) function
equalObjects(a, b), with the following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also deals
with null references. What do you think?
Andrei
I like this. I think optimizing away opEquals for identical objects
would also be a good idea:
static if (is(U == class))
if(a is b || a is null || b is null) return a is b;
else
enforce(a !is null);
This code has an inefficiency, it seems, because it makes a bit more
checks than necessary (e.g. checks a is b twice). Let's simplify:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (a is b) return true;
if (b is null || a is null) return false;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
Andrei
Re: The Non-Virtual Interface idiom in D
On 2009-09-26 10:06:24 -0400, Andrei Alexandrescu said: Michel Fortin wrote: I think you're writing a lot of boilerplate code for something that the compiler should be able to do by itself. I mean, it's a lot cleaner with contracts, and there is no reason the compiler couldn't generate itself that "contract-verifying" non-virtual function. I think it would be a mistake to latch on my quick examples. It's not only about before and after checks, it's more about low-level customization points versus higher-level interfaces. Then your examples should have shown this instead. I fully support having a way to specify a default implementation for a function in an interface. It might get handy for a few things (like implementing the delegate pattern you see everywhere in Cocoa). But it's a bad replacement for contracts. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Null references redux
downs: > Basically, anything that may fill it with nulls. > > The only two allowed instructions would be ~= NonNullable and ~= > NonNullableArray. And it's good that way. I agree. In such situation I'd also like to have a default method to insert one or more nonnull items in any point of the array (see insert method of Python lists, that can also be expressed as s[i:i]=[x]). Having fee basic default methods will help keep such safe arrays flexible. Bye, bearophile
Re: putting more smarts into a == b
Jarrett Billingsley wrote:
On Sat, Sep 26, 2009 at 9:32 PM, Andrei Alexandrescu
wrote:
Consider two objects a and b with a of class type. Currently, the expression
a == b is blindly rewritten as a.opEquals(b). I argue it should be rewritten
into a call to an (imaginary/inlined) function equalObjects(a, b), with the
following definition:
bool equalObjects(T, U)(T a, U b) if (is(T == class))
{
static if (is(U == class))
{
if (b is null) return a is null;
if (a is null) return b is null;
}
else
{
enforce(a !is null);
}
return a.opEquals(b);
}
This hoists the identity test outside the opEquals call and also deals with
null references. What do you think?
I'm almost sure that C# does this already, and it's a useful behavior.
C# operator overloads are of the form:
public static ReturnType operator+(Arg1 arg1, Arg2 arg2) {}
Object.operator== is defined to call arg1.Equals(arg2) if arg1 isn't
null. But this isn't a feature of operator overloads.
Of course, with nonnull types, the check for null wouldn't even need
to exst ;)
How clever and insightful of you!
Re: Null references redux
On 2009-09-27 07:38:59 -0400, Christopher Wright said: I dislike these forced checks. Let's say you're dealing with a compiler frontend. You have a semantic node that just went through some semantic pass and is guaranteed, by flow control and contracts, to have a certain property initialized that was not initialized prior to that point. The programmer knows the value isn't null. The compiler shouldn't force checks. At most, it should have automated checks that disappear with -release. If the programmer knows a value isn't null, why not put the value in a nullable-reference in the first place? Also, it introduces more nesting. Yes and no. It introduces an "if" statement for null checking, but only for nullable references. If you know your reference can't be null it should be non-nullable, and then you don't need to check. Also, unless the compiler's flow analysis is great, it's a nuisance -- you can see that the error is bogus and have to insert extra checks. First you're right, if the feature is implemented it should be well implemented. Second, if in a few place you don't want an "if" clause, you can always cast your nullable reference to a non-nullable one, explicitly bypassing the safeties. If you write a cast, you are making a consious decision of not checking for null, which is much better than the current situation where it's very easy to forget to check for null. It should be fine to provide a requireNotNull template and leave it at that. It's fine to have such a template. But it's not nearly as useful. -- Michel Fortin michel.for...@michelf.com http://michelf.com/
Re: Null references redux
Walter Bright wrote: > Name any single part or system on a Boeing airliner, and if it > vanishes abruptly in a puff of smoke, the airliner will survive it. Except this sentence I applaud every thought. If "single part" includes the passenger area, the meaning of this sentence is upright ridiculous. -manfred
Re: Null references redux
Michel Fortin wrote: On 2009-09-26 22:07:00 -0400, Walter Bright said: [...] The facilities in D enable one to construct a non-nullable type, and they are appropriate for many designs. I just don't see them as a replacement for *all* reference types. As far as I understand this thread, no one here is arguing that non-nullable references/pointers should replace *all* reference/pointer types. The argument made is that non-nullable should be the default and nullable can be specified explicitly any time you need it. So if you need a reference you use "Object" as the type, and if you want that reference to be nullable you write "Object?". The static analysis can then assert that your code properly check for null prior dereferencing a nullable type and issues a compilation error if not. I dislike these forced checks. Let's say you're dealing with a compiler frontend. You have a semantic node that just went through some semantic pass and is guaranteed, by flow control and contracts, to have a certain property initialized that was not initialized prior to that point. The programmer knows the value isn't null. The compiler shouldn't force checks. At most, it should have automated checks that disappear with -release. Also, it introduces more nesting. Also, unless the compiler's flow analysis is great, it's a nuisance -- you can see that the error is bogus and have to insert extra checks. It should be fine to provide a requireNotNull template and leave it at that.
Re: Null references redux
Andrei Alexandrescu wrote: Walter Bright wrote: Justin Johansson wrote: Walter, in the heat of this thread I hope you haven't missed the correlation with discussion on "Dispatching on a variant" and noting: Thanks for pointing it out. The facilities in D enable one to construct a non-nullable type, and they are appropriate for many designs. No. There is no means to disable default construction. I looked into this slightly. You'd have to do mark non-nullable fields as requiring ctor initialization, prevent reallocating arrays of non-nullables, and a few other things. At the time I wasn't considering struct constructors; without them, you'd have to forbid structs that contain non-nullable fields, but with them, it's okay. I just don't see them as a replacement for *all* reference types. Non-nullable references should be the default. Andrei
Re: Null references redux
Jeremie Pelletier wrote: There again, I favor stronger initialization semantics over nonnull types. This will get rid of most of these errors Only for local variables. Not for fields. Most segfaults I have take me at most a few minutes to pinpoint. Its finding backdoors to compiler enforcements thats annoying. You're complaining now because you'd try to cram 'null' down the throat of something marked 'not-null' and fear it would be difficult?
Re: Null references redux
Walter Bright wrote: Andrei Alexandrescu wrote: Walter Bright wrote: Even forcing an explicit initializer doesn't actually solve the problem - my experience with such features is programmers simply insert any old value to get the code to pass the compiler, even programmers who know it's a bad idea do it anyway. I think you're starting to be wrong at the point where you don't realize that many bugs come from references that people have forgotten to initialize. Once you acknowledge those, you will start to realize that a reference that must compulsively be initialized is valuable. The problem is it's worse to force people to provide an initializer. You aren't forcing them. They decide for themselves. They determine whether it's appropriate for a particular variable to be null. You can achieve the same goal through contracts. However, this is much more verbose -- enough so that you'll only add these contracts when hunting down a bug. And if you have an array of things It isn't a theoretical problem with providing bad initializers just to shut the compiler up. I have seen it in the wild every time some manager required that code compile without warnings and the compiler warned about no initializer. C# requires that every variable be initialized before use. You know how often I get such an error? Maybe once for every 100 hours of coding. It's mainly for cases where I expect an integer to be initialized to 0 and it's not. You know how often I provide a bad initializer to shut the compiler up? Never. This is partially because C#'s compiler has good flow analysis. It's mostly because: - I declare variables where I use them, not beforehand. - I often declare variables via IDE commands -- I write the code to fetch or calculate a value and assign it to a variable that doesn't exist, and the IDE fills in the type and declares it in the correct place. - I usually don't have more than four or five local variables in a function (often no more than one or two). Out of 300KLOC, there are a few dozen functions that break this rule. DMDFE functions are often long, complex, and have many local variables. I see how this would conflict with your coding style. You would have to add a few question marks for each function, and then you'd be done. DMDFE is ~60KLOC, but you could probably switch it over to this type system without structural changes to any function in a couple days.
Re: Null references redux
Sun, 27 Sep 2009 00:27:14 -0700, Walter Bright thusly wrote: >> You seem to be under the impression that nothing can be made >> uncrashable without introducing the possibility of corrupted state. >> That's hogwash. What I mean by safe is that no matter what you do, you cannot make the program crash or cause memory corruption. If you look at typical functional languages, unless FFI is used, the only ways the program may fail are a) no more stack memory b) no more heap memory c) programs halts (halting problem) d) developer explicitly kills the program e.g. with the Error type. Note that if your language is simple enough, say simply typed lambda calculus, you do not have the third problem anymore. All of these errors can also happen in D, but none of the D's other problems happen in those languages.
Re: Null references redux
Jeremie Pelletier wrote:
> Christopher Wright wrote:
>> Jeremie Pelletier wrote:
>>> What if using 'Object obj;' raises a warning "unitialized variable"
>>> and makes everyone wanting non-null references happy, and 'Object obj
>>> = null;' raises no warning and makes everyone wanting to keep the
>>> current system (all two of us!) happy.
>>>
>>> I believe it's a fair compromise.
>>
>> It's a large improvement, but only for local variables. If your
>> segfault has to do with a local variable, unless your function is
>> monstrously large, it should be easy to fix, without changing the type
>> system.
>>
>> The larger use case is when you have an aggregate member that cannot
>> be null. This can be solved via contracts, but they are tedious to
>> write and ubiquitous.
>
> But how would you enforce a nonnull type over an aggregate in the first
> place? If you can, you could also apply the same initializer semantics I
> suggested earlier.
>
> Look at this for example:
>
> struct A {
> Object cannotBeNull;
> }
>
> void main() {
> A* a = new A;
> }
>
> Memory gets initialized to zero, and you have a broken non-null type.
> You could have the compiler throw an error here, but the compiler cannot
> possibly know about all data creation methods such as malloc, calloc or
> any other external allocator.
>
> You could even do something like:
>
> Object* foo = calloc(Object.sizeof);
>
> and the compiler would let you dereference foo resulting in yet another
> broken nonnull variable.
>
> Non-nulls are a cute idea when you have a type system that is much
> stricter than D's, but there are just way too many workarounds to make
> it crash in D.
"Here are some cases you haven't mentioned yet. This proves that the compiler
can't possibly be smart enough. "
Yeah.
In the above case, why not implicitly put the cannotBeNull check into the
struct invariant? That's where it belongs, imho.
Regarding your example, it's calloc(size_t.sizeof). And a) we probably can't
catch that case except with in/out null checks on every method, but then again,
how often have you done that? I don't think it's relevant enough to be relevant
to this thread. :)
Re: Null references redux
Jeremie Pelletier wrote: > Jarrett Billingsley wrote: >> On Sat, Sep 26, 2009 at 11:23 PM, Jeremie Pelletier >> wrote: >> >>> There is no such thing as "not being able to happen" :) >>> >>> Object thisCannotPossiblyBeNullInAnyWayWhatsoever = cast(Object)null; >>> >>> I seem to be the only one who sees Walter's side of things in this >>> thread >>> :o) >> >> Why the hell would the compiler allow that to begin with? Why bother >> implementing nonnull references only to allow the entire system to be >> broken? > > Because D is a practical language that let the programmer do whatever he > wants, even shoot his own foot if he wants to. Doing so just isn't as > implicit as in C. > > Walter understands there are some cases where you want to override the > type system, that's why casts are in D, too many optimizations rely on it. Sure, but if you set out to break it the compiler really can't (or shouldn't) help you. This whole debate, as far as I know, is about defaults, i.e. preventing *unintentional* nulls.
Re: Null references redux
Denis Koroskin wrote:
> On Sun, 27 Sep 2009 03:01:48 +0400, Walter Bright
> wrote:
>
>> Denis Koroskin wrote:
>>> One more:
>>> T foo(bool someCondition)
>>> {
>>> T? t;
>>> if (someCondition) t = someInitializer();
>>> // ...
>>> if (t.isNull) { // not initialized yet
>>> // ...
>>> }
>>> return enforce(t); // throws if t is not initialized yet,
>>> because foo *must* return a valid value by a contract
>>> }
>>
>> It seems to me you've got null references there anyway?
>>
>> What would you do about:
>>
>> T[] a;
>> a[i] = foo();
>>
>> where you want to have unused slots be null (or empty, or nothing)?
>
> Easy:
>
> T? foo(); // returns valid object or a null
>
> T?[] a;
> a[i] = foo();
The case of a non-null array is, I think, worthy of some more consideration.
These are the things that would not be possible with a non-nullable array:
- newing it
- setting .length to a greater value
- appending a nullable array of the same base type.
Basically, anything that may fill it with nulls.
The only two allowed instructions would be ~= NonNullable and ~=
NonNullableArray. And it's good that way.
Re: Null references redux
Sun, 27 Sep 2009 00:08:50 -0400, Jeremie Pelletier thusly wrote: > Ary Borenszweig wrote: >> Just out of curiosity: have you ever programmed in Java or C#? > > Nope, never got interested in these to tell the truth. I only did C, > C++, D and x86 assembly in systems programming, I have quite a > background in PHP and JavaScript also. So you only know imperative procedural programming + some features of hybrid OOP languages that are not even proper OOP languages. > > I played with a lot of languages, but those are the ones I use on a > daily basis. I would like to get into Python or Ruby someday, I only > hear good things about these two. I know LUA has less overhead than > Python Oh, the only difference between LUA and Python is the overhead?! That's a... pretty performance oriented view on languages. > I like extremes :) If you like extremes, why have you not programming in Haskell or Coq? Too scary? You are often arguing against languages and concepts you have never used. The other people here who make these suggestions are more experienced with various languages.
Re: Null references redux
Walter Bright wrote: > Nick Sabalausky wrote: > > I agree with you that if the compiler can detect null dereferences at > compile time, it should. > > >>> Also, by "safe" I presume you mean "memory safe" which means free of >>> memory corruption. Null pointer exceptions are memory safe. A null >>> pointer could be caused by memory corruption, but it cannot *cause* >>> memory corruption. >> >> No, he's using the real meaning of "safe", not the >> misleadingly-limited "SafeD" version of "safe" (which I'm still >> convinced is going to get some poor soul into serious trouble from >> mistakingly thinking their SafeD program is much safer than it really >> is). Out here in reality, "safe" also means a lack of ability to >> crash, or at least some level of protection against it. > > Memory safety is something that can be guaranteed (presuming the > compiler is correctly implemented). There is no way to guarantee that a > non-trivial program cannot crash. It's the old halting problem. > Okay, I'm gonna have to call you out on this one because it's simply incorrect. The halting problem deals with a valid program state - halting. We cannot check if every program halts because halting is an instruction that must be allowed at almost any point in the program. Why do crashes have to be allowed? They're not an allowed instruction! A compiler can be turing complete and still not allow crashes. There is nothing wrong with this, and it has *nothing* to do with the halting problem. >> You seem to be under the impression that nothing can be made >> uncrashable without introducing the possibility of corrupted state. >> That's hogwash. > > I read that statement several times and I still don't understand what it > means. > > BTW, hardware null pointer checking is a safety feature, just like array > bounds checking is. PS: You can't convert segfaults into exceptions under Linux, as far as I know.
Re: Null references redux
Jeremie Pelletier wrote:
> void bar(bool foo) {
> if(foo) {
> int a = 1;
> ...
> }
> else {
> int a = 2;
> ...
> }
>
> }
>
> is the stack frame using two ints, or is the compiler seeing only one? I
> never bothered to check it out and just declared 'int a = void;' at the
> beginning of the routine to keep the stack frames as small as possible.
OT, but declaring the variable at the top of the function increases
stack size.
Example with changed variable names:
void bar(bool foo) {
if (foo) {
int a = 1;
} else {
int b = 2;
}
int c = 3;
}
In this example, there are clearly three different (and differently
named) variables, but their lifetimes do not overlap. Only one variable
can exist at a time, therefore the compiler only needs to allocate space
for one variable. Now, if you move your declaration to the top:
void bar(bool foo) {
int a = void;
if (foo) {
a = 1;
} else {
a = 2; // Reuse variable.
}
int c = 3;
}
You now only have two variables, but both of them coexist at the end of
the function. Unless the compiler applies a clever optimization, the
compiler is now forced to allocate space for two variables on the stack.
--
Rainer Deyke - rain...@eldwood.com
Re: Null references redux
Jeremie Pelletier wrote:
This may be a good time to ask about how these variables which can be
declared anywhere in the function scope are implemented.
void bar(bool foo) {
if(foo) {
int a = 1;
...
}
else {
int a = 2;
...
}
}
is the stack frame using two ints, or is the compiler seeing only one? I
never bothered to check it out and just declared 'int a = void;' at the
beginning of the routine to keep the stack frames as small as possible.
They are completely independent variables. One may get assigned to a
register, and not the other.
