Re: Null references (oh no, not again!)

[Rainer Deyke]

Daniel Keep wrote:
> The point was that these were identified as being responsible for the
> majority of the bugs in a large, real-world code base.  Clearly #2 and
> #3 are common enough and cause enough issues to have made the list.

A sample size of one doesn't mean much.  In my experience, none of those
four factors account for a significant amount of bugs, since all of them
(except integer overflow) can be caught without too much effort through
the copious use of assertions.

I'd still prefer non-nullable references to be the default though.
Writing an assertion for every non-nullable reference argument for every
function is tedious.


-- 
Rainer Deyke - rain...@eldwood.com

Re: Null references (oh no, not again!)

[Daniel Keep]

Walter Bright wrote:
> Daniel Keep wrote:
>>> * Accessing arrays out-of-bounds
>>> * Dereferencing null pointers
>>> * Integer overflow
>>> * Accessing uninitialized variables
>>>
>>> 50% of the bugs in Unreal can be traced to these problems!
>>
>> Tim Sweeny isn't an amateur; he's responsible, at least in part, for one
>> of the most commercially successful game engines ever.  I figure if even
>> he has trouble with these things, it's worth trying to fix them.
>>
>> Note that D already solves #1 and #4, LDC could give us #3... that just
>> leaves #2.  :D
> 
> 1 and 4 are pernicious, memory corrupting, hard to find problems. 2 is
> easy to find, does not corrupt memory. It isn't even in the same
> continent as 1 and 4 are.
> 
> 3 is a problem, but fortunately it tends to be rare.

The point was that these were identified as being responsible for the
majority of the bugs in a large, real-world code base.  Clearly #2 and
#3 are common enough and cause enough issues to have made the list.

  -- Daniel

Re: const?? When and why? This is ugly!

[Denis Koroskin]

On Wed, 04 Mar 2009 08:55:18 +0300, novice2  wrote:


Denis Koroskin Wrote:


// file 1
char[] a = "hello";
a[4] = '!';

// file 2
writefln("hello"); // prints 'hell!'



bad example :)
why so behaviour? imho, it may be occur bacause of bad compiler design  
only. like C compiler for space optimization collect all mentioning of  
similar strings and allocate only one for all of occurence. and even in  
this case it must allocate it in non-writable memory section.


Yeah, that's why it is forbidden. It won't compile in D2.

Re: Null references (oh no, not again!)

[Walter Bright]

Jarrett Billingsley wrote:

Exactly.  I thought one of the ideas behind D was to have "safe"
defaults.  Yeah, I know, null references can't actually do damage to
your computer because of virtual memory, but neither can concurrent
access to shared data, or accessing uninitialized variables, but
they're taken care of.


Those last two *are* unsafe, memory corrupting problems.

Re: const?? When and why? This is ugly!

[novice2]

Denis Koroskin Wrote:

> // file 1
> char[] a = "hello";
> a[4] = '!';
> 
> // file 2
> writefln("hello"); // prints 'hell!'


bad example :)
why so behaviour? imho, it may be occur bacause of bad compiler design only. 
like C compiler for space optimization collect all mentioning of similar 
strings and allocate only one for all of occurence. and even in this case it 
must allocate it in non-writable memory section.

Re: Null references (oh no, not again!)

[Walter Bright]

Daniel Keep wrote:

* Accessing arrays out-of-bounds
* Dereferencing null pointers
* Integer overflow
* Accessing uninitialized variables

50% of the bugs in Unreal can be traced to these problems!


Tim Sweeny isn't an amateur; he's responsible, at least in part, for one
of the most commercially successful game engines ever.  I figure if even
he has trouble with these things, it's worth trying to fix them.

Note that D already solves #1 and #4, LDC could give us #3... that just
leaves #2.  :D


1 and 4 are pernicious, memory corrupting, hard to find problems. 2 is 
easy to find, does not corrupt memory. It isn't even in the same 
continent as 1 and 4 are.


3 is a problem, but fortunately it tends to be rare.

Re: Null references (oh no, not again!)

[Michel Fortin]

On 2009-03-03 13:59:16 -0500, Andrei Alexandrescu 
 said:


I suggested to Walter an idea he quite took to: offering the ability of 
disabling the default constructor. This is because at root any null 
pointer was a pointer created with its default constructor. The feature 
has some interesting subtleties to it but is nothing out of the 
ordinary and the code must be written anyway for typechecking invariant 
constructors.


That, together with the up-and-coming alias this feature, will allow 
the creation of the "perfect" NonNull!(T) type constructor (along with 
many other cool things). I empathize with those who think non-null 
should be the default, but probably that won't fly with Walter.


That'd be great, really.

But even then, NonNull!(T) will probably be to D what auto_ptr< T > is 
to C++: a very good idea with a very bad syntax only expert programmers 
use. C++ makes the safest pointer types the less known; please convince 
Walter we shouldn't repeat that error in D.



--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: const?? When and why? This is ugly!

[hasen]

Walter Bright wrote:

grauzone wrote:
I didn't mean going back to programming with locks. Instead you could 
use the new ideas without extending the type system. As far as I 
understand, the language extensions are only needed for verification 
(so far).


Without verification, it's programming by hopeful convention. If you 
want a reliable system, you need more than hope .


Well .. if you think about OOP and private/public ..

Dynamic languages like python and smalltalk don't enforce 
private/public, and that never was a problem. And, smalltalk is *the* OO 
language (AFAIK)


(this is not really an argument against const per se, it's just an 
argument against an argument for const)

Re: Null references (oh no, not again!)

[Daniel Keep]

Andrei Alexandrescu wrote:
> I suggested to Walter an idea he quite took to: offering the ability of
> disabling the default constructor. This is because at root any null
> pointer was a pointer created with its default constructor. The feature
> has some interesting subtleties to it but is nothing out of the ordinary
> and the code must be written anyway for typechecking invariant
> constructors.
> 
> That, together with the up-and-coming alias this feature, will allow the
> creation of the "perfect" NonNull!(T) type constructor (along with many
> other cool things). I empathize with those who think non-null should be
> the default, but probably that won't fly with Walter.
> 
> 
> Andrei

If Walter can get this working so that we can create NonNull as a
wrapper struct that guarantees it will always be non-null, that'll be a
big step forward.

Yes, I'd prefer it be the default, but better possible than not.  :)

  -- Daniel

Re: Null references (oh no, not again!)

[Jason House]

Andrei Alexandrescu Wrote:

> Daniel Keep wrote:
> > Just noticed this hit Slashdot, and thought I might repost the abstract
> > here.
> > 
> > http://qconlondon.com/london-2009/presentation/Null+References:+The+Billion+Dollar+Mistake
> > 
> >> I call it my billion-dollar mistake. It was the invention of the null
> >> reference in 1965. [...] This has led to innumerable errors,
> >> vulnerabilities, and system crashes, which have probably caused a
> >> billion dollars of pain and damage in the last forty years. [...] More
> >> recent programming languages like Spec# have introduced declarations
> >> for non-null references. This is the solution, which I rejected in
> >> 1965.
> > 
> >   -- Sir Charles Hoare, Inventor of QuickSort, Turing Award Winner
> 
> I suggested to Walter an idea he quite took to: offering the ability of 
> disabling the default constructor. This is because at root any null 
> pointer was a pointer created with its default constructor. The feature 
> has some interesting subtleties to it but is nothing out of the ordinary 
> and the code must be written anyway for typechecking invariant 
> constructors.
> 
> That, together with the up-and-coming alias this feature, will allow the 
> creation of the "perfect" NonNull!(T) type constructor (along with many 
> other cool things). I empathize with those who think non-null should be 
> the default, but probably that won't fly with Walter.
> 
> 
> Andrei

Alias this?

Re: Null references (oh no, not again!)

[bearophile]

Christopher Wright:
>I'm still sad about mutable being the default in d2.<

Maybe D3 will move more in that direction, I don't know. It's a really big jump 
from C/C++, quite bigger than nonnullable by default :-)

Bye,
bearophile

Re: Null references (oh no, not again!)

[Christopher Wright]

Daniel Keep wrote:

Just noticed this hit Slashdot, and thought I might repost the abstract
here.

http://qconlondon.com/london-2009/presentation/Null+References:+The+Billion+Dollar+Mistake


I call it my billion-dollar mistake. It was the invention of the null
reference in 1965. [...] This has led to innumerable errors,
vulnerabilities, and system crashes, which have probably caused a
billion dollars of pain and damage in the last forty years. [...] More
recent programming languages like Spec# have introduced declarations
for non-null references. This is the solution, which I rejected in
1965.


  -- Sir Charles Hoare, Inventor of QuickSort, Turing Award Winner

Serendipitous, since I just spent today trying to track down an
(expletives deleted) obscure null dereference problem.  I figure I must
be in good company if even the guy who invented null doesn't like it...


There are issues shoe-horning non-nullables into a nullable world:
 - preallocating arrays (or static arrays)
 - structs with non-nullable fields
 - pointers to non-nullables

It's sufficient that I gave up on my attempts to implement it.

If it were implemented, non-nullable absolutely must be the default. I'm 
still sad about mutable being the default in d2.

Re: std.locale

[Christopher Wright]

Walter Bright wrote:

Georg Wrede wrote:
We've had Walter make nice features to D that were laborious to 
create, only to see nobody use them. It's happened, ask him.


Sure. Often the only way to see if a feature is useful is to actually 
implement it and see what happens. Some features have succeeded and 
found uses far beyond my expectations (CTFE, string mixins) while others 
have pretty much languished (design by contract, complex numbers).


I fucking love contracts. I need to use them more, but I do use them.

Re: Compilation of .di needes the imported file and -J setup correctly

[Qian Xu]

Frank Benoit wrote:

When doing incremental compilation .di shall help. As I understand it,
it should be possible to compile the modules for a lib and provide the
object files or a library and the .di files.

When doing this, i encounter problems, because now, the user code also
needs the files from the libs setup correctly in the -J path. This makes
the build process more complex. naming conflicts can occur.

To solve this, I want to suggest, that at .di generation the
text-imported files are inserted into the generated .di as a literal. So
the .di file can stand for there own, without the need for any -J option.


I feel it strange. The header files I have made (with gdc in Linux) is 
almost the same as the source code (include the implementation).



--
Xu, Qian (stanleyxu)
 http://stanleyxu2005.blogspot.com

Re: Null references (oh no, not again!)

[Brad Roberts]

On Tue, 3 Mar 2009, Andrei Alexandrescu wrote:

> I did some more research and found a study:
> 
> http://users.encs.concordia.ca/~chalin/papers/TR-2006-003.v3s-pub.pdf
> 
> Very interestingly (and exactly the kind of info I was looking for), the study
> measures how references are meant to be in a real application of medium-large
> size.
> 
> Turns out in 2/3 of cases, references are really meant to be non-null... not
> really a landslide but a comfortable majority.
> 
> 
> Andrei

I'd love to see a similar study for smart pointers.  Are they more like 
pointers or references?  My assumption is references, leading to them 
beign poorly named. :)

Later,
Brad

Re: Null references (oh no, not again!)

[Andrei Alexandrescu]

Denis Koroskin wrote:
On Tue, 03 Mar 2009 21:59:16 +0300, Andrei Alexandrescu 
 wrote:



Daniel Keep wrote:

Just noticed this hit Slashdot, and thought I might repost the abstract
here.
 http://qconlondon.com/london-2009/presentation/Null+References:+The+Billion+Dollar+Mistake 




I call it my billion-dollar mistake. It was the invention of the null
reference in 1965. [...] This has led to innumerable errors,
vulnerabilities, and system crashes, which have probably caused a
billion dollars of pain and damage in the last forty years. [...] More
recent programming languages like Spec# have introduced declarations
for non-null references. This is the solution, which I rejected in
1965.

   -- Sir Charles Hoare, Inventor of QuickSort, Turing Award Winner


I suggested to Walter an idea he quite took to: offering the ability 
of disabling the default constructor. This is because at root any null 
pointer was a pointer created with its default constructor. The 
feature has some interesting subtleties to it but is nothing out of 
the ordinary and the code must be written anyway for typechecking 
invariant constructors.


That, together with the up-and-coming alias this feature, will allow 
the creation of the "perfect" NonNull!(T) type constructor (along with 
many other cool things). I empathize with those who think non-null 
should be the default, but probably that won't fly with Walter.



Andrei


If nullable is the default and NonNull!(T) has no syntactic sugar, I bet 
it won't be used at all. I know I woudn't, even though I'm one of the 
biggest advocates of introducing non-nullable types in D.


In my opinion, you should teach novices safe practices first, and 
dangerous tricks last. Not vice-versa.


If using of nullable types would be easier that non-nullable once, it 
won't be widely used. The syntax ought to be less verbose and more clear 
to get an attention.


I hope that this great idea won't get spoiled by broken implementation...



I did some more research and found a study:

http://users.encs.concordia.ca/~chalin/papers/TR-2006-003.v3s-pub.pdf

Very interestingly (and exactly the kind of info I was looking for), the 
study measures how references are meant to be in a real application of 
medium-large size.


Turns out in 2/3 of cases, references are really meant to be non-null... 
not really a landslide but a comfortable majority.



Andrei

Compilation of .di needes the imported file and -J setup correctly

[Frank Benoit]

When doing incremental compilation .di shall help. As I understand it,
it should be possible to compile the modules for a lib and provide the
object files or a library and the .di files.

When doing this, i encounter problems, because now, the user code also
needs the files from the libs setup correctly in the -J path. This makes
the build process more complex. naming conflicts can occur.

To solve this, I want to suggest, that at .di generation the
text-imported files are inserted into the generated .di as a literal. So
the .di file can stand for there own, without the need for any -J option.

Re: std.locale

[Derek Parnell]

On Tue, 03 Mar 2009 11:00:36 -0800, Walter Bright wrote:

> Contracts are not for input validation!

Hear! Hear! This is exactly correct.


-- 
Derek Parnell
Melbourne, Australia
skype: derek.j.parnell

Re: The Sweet With

[Jarrett Billingsley]

On Tue, Mar 3, 2009 at 3:32 PM, Nick Sabalausky  wrote:
>> "Jarrett Billingsley"  wrote in message
>> news:mailman.901.1236111433.22690.digitalmar...@puremagic.com...
>> On Tue, Mar 3, 2009 at 1:50 PM, Walter Bright
>>  wrote:
>> > Tomasz Sowiński wrote:
>> >>
>> >> Ideas for features based on the with.
>> >>
>> >> The with can make calling functions with enum arguments sexier. So
>> >> instead
>> >> of:
>> >> auto d = dirEntries(".", SpanMode.breadth);
>> >>
>> >> you could say:
>> >> auto d = dirEntries(".", breadth);
>> >>
>> >> by declaring the function as:
>> >> dirEntries(string path, with SpanMode mode); // "with" does the trick
>> >
>> > It looks nice, but has a subtle and disastrous problem. In D, arguments
>> > are
>> > fully resolved *before* overloading is done. If some of the overloads
>> > have
>> > with declarations, then there's a nightmarish problem of trying to mix
>> > overloading and argument resolution together.
>>
>> What about the feature you mentioned at the D con, about being able to
>> use enums without the enum name?  Or will/would that only be for
>> things where it's really obvious, like switch statements?
>
> Unless that was only for things like switch statements, I would hate that.
> I've used enums in languages that worked that way, and I found it to be such
> a problematic namespace-clutterer that in those languages I always hack up
> my enum definitions like this:
>
> enum Color
> {
>    Color_Red,
>    Color_Blue,
>    Color_Orange,
>    // etc...
> }
>
> Which is a style that I've always considered an ugly and kludgey, but
> unfortunately necessary, substitute for manditory enum names.

Oh, the way he described it in the slides was different.  As in,
normally you would use Color.Red, but inside a switch:

switch(widgetColor)
{
case Red: // this is *completely* unambiguous
}

And such.  Red is not a global; it's rather that name lookup is
changed for some constructs.

Re: The Sweet With

[Walter Bright]

Jarrett Billingsley wrote:

What about the feature you mentioned at the D con, about being able to
use enums without the enum name?  Or will/would that only be for
things where it's really obvious, like switch statements?


As I recall, only some of that would work because of this problem.

Re: std.locale

[Sean Kelly]

== Quote from Walter Bright (newshou...@digitalmars.com)'s article
> Andrei Alexandrescu wrote:
> > I agree. I'm having the same problem: I put a contract in there, I know
> > it's as good as assert. So I can't do e.g. input validation because in
> > most functions input must always be validated. I also know that
> > contracts are doing the wrong thing with inheritance and can't apply to
> > interfaces, which is exactly the (only?) place they'd be interesting. So
> > I send the contracts home and use assert, enforce, and unittest.
> Contracts are not for input validation! They are checking if the logic
> of your program is correct or not. Think of it this way - your program
> should behave exactly the same with or without the contracts turned on.
> Contracts should NOT be used for scrubbing user input, checking for
> errors from other components, or validating any input from external to
> the dll.

Why should contracts be limited to parameter checking of internally used
functions only?  If I write a function and document parameter constraints
then I certainly expect those constraints to be followed regardless of
whether I'm calling the function or someone else is calling the function.
Checking these via a contract simply provides an optional means of
ensuring that a logic error didn't occur within the program as a whole.

If you're talking about application input however, then I agree completely.
ie. stuff typed in by the user, read from a file, etc, should never be validated
within a contract because an input failure at that level doesn't represent
a program logic error but rather user error.  An assertion failure isn't
a terribly good way of notifying the user that they shouldn't have put an
alphabetic character in a box intended to receive an integer :-)


Sean

Re: The Sweet With

[Nick Sabalausky]

> "Jarrett Billingsley"  wrote in message 
> news:mailman.901.1236111433.22690.digitalmar...@puremagic.com...
> On Tue, Mar 3, 2009 at 1:50 PM, Walter Bright
>  wrote:
> > Tomasz Sowiński wrote:
> >>
> >> Ideas for features based on the with.
> >>
> >> The with can make calling functions with enum arguments sexier. So 
> >> instead
> >> of:
> >> auto d = dirEntries(".", SpanMode.breadth);
> >>
> >> you could say:
> >> auto d = dirEntries(".", breadth);
> >>
> >> by declaring the function as:
> >> dirEntries(string path, with SpanMode mode); // "with" does the trick
> >
> > It looks nice, but has a subtle and disastrous problem. In D, arguments 
> > are
> > fully resolved *before* overloading is done. If some of the overloads 
> > have
> > with declarations, then there's a nightmarish problem of trying to mix
> > overloading and argument resolution together.
>
> What about the feature you mentioned at the D con, about being able to
> use enums without the enum name?  Or will/would that only be for
> things where it's really obvious, like switch statements?

Unless that was only for things like switch statements, I would hate that. 
I've used enums in languages that worked that way, and I found it to be such 
a problematic namespace-clutterer that in those languages I always hack up 
my enum definitions like this:

enum Color
{
Color_Red,
Color_Blue,
Color_Orange,
// etc...
}

Which is a style that I've always considered an ugly and kludgey, but 
unfortunately necessary, substitute for manditory enum names.

Re: Null references (oh no, not again!)

[Jarrett Billingsley]

On Tue, Mar 3, 2009 at 3:08 PM, Denis Koroskin <2kor...@gmail.com> wrote:
>
> If nullable is the default and NonNull!(T) has no syntactic sugar, I bet it
> won't be used at all. I know I woudn't, even though I'm one of the biggest
> advocates of introducing non-nullable types in D.
>
> In my opinion, you should teach novices safe practices first, and dangerous
> tricks last. Not vice-versa.

Exactly.  I thought one of the ideas behind D was to have "safe"
defaults.  Yeah, I know, null references can't actually do damage to
your computer because of virtual memory, but neither can concurrent
access to shared data, or accessing uninitialized variables, but
they're taken care of.

If nonnull types were the default, Nullable!(T) would be implementable
as an Algebraic type, just like in Haskell.  One more potential
bragging point ;)

Re: The Sweet With

[Jarrett Billingsley]

On Tue, Mar 3, 2009 at 1:50 PM, Walter Bright
 wrote:
> Tomasz Sowiński wrote:
>>
>> Ideas for features based on the with.
>>
>> The with can make calling functions with enum arguments sexier. So instead
>> of:
>> auto d = dirEntries(".", SpanMode.breadth);
>>
>> you could say:
>> auto d = dirEntries(".", breadth);
>>
>> by declaring the function as:
>> dirEntries(string path, with SpanMode mode);    // "with" does the trick
>
> It looks nice, but has a subtle and disastrous problem. In D, arguments are
> fully resolved *before* overloading is done. If some of the overloads have
> with declarations, then there's a nightmarish problem of trying to mix
> overloading and argument resolution together.

What about the feature you mentioned at the D con, about being able to
use enums without the enum name?  Or will/would that only be for
things where it's really obvious, like switch statements?

Re: Null references (oh no, not again!)

[Denis Koroskin]

On Tue, 03 Mar 2009 21:59:16 +0300, Andrei Alexandrescu 
 wrote:


Daniel Keep wrote:

Just noticed this hit Slashdot, and thought I might repost the abstract
here.
  
http://qconlondon.com/london-2009/presentation/Null+References:+The+Billion+Dollar+Mistake



I call it my billion-dollar mistake. It was the invention of the null
reference in 1965. [...] This has led to innumerable errors,
vulnerabilities, and system crashes, which have probably caused a
billion dollars of pain and damage in the last forty years. [...] More
recent programming languages like Spec# have introduced declarations
for non-null references. This is the solution, which I rejected in
1965.

   -- Sir Charles Hoare, Inventor of QuickSort, Turing Award Winner


I suggested to Walter an idea he quite took to: offering the ability of  
disabling the default constructor. This is because at root any null  
pointer was a pointer created with its default constructor. The feature  
has some interesting subtleties to it but is nothing out of the ordinary  
and the code must be written anyway for typechecking invariant  
constructors.


That, together with the up-and-coming alias this feature, will allow the  
creation of the "perfect" NonNull!(T) type constructor (along with many  
other cool things). I empathize with those who think non-null should be  
the default, but probably that won't fly with Walter.



Andrei


If nullable is the default and NonNull!(T) has no syntactic sugar, I bet it 
won't be used at all. I know I woudn't, even though I'm one of the biggest 
advocates of introducing non-nullable types in D.

In my opinion, you should teach novices safe practices first, and dangerous 
tricks last. Not vice-versa.

If using of nullable types would be easier that non-nullable once, it won't be 
widely used. The syntax ought to be less verbose and more clear to get an 
attention.

I hope that this great idea won't get spoiled by broken implementation...

Re: std.locale

[Max Samukha]

On Tue, 03 Mar 2009 11:00:36 -0800, Walter Bright
 wrote:

>Andrei Alexandrescu wrote:
>> I agree. I'm having the same problem: I put a contract in there, I know 
>> it's as good as assert. So I can't do e.g. input validation because in 
>> most functions input must always be validated. I also know that 
>> contracts are doing the wrong thing with inheritance and can't apply to 
>> interfaces, which is exactly the (only?) place they'd be interesting. So 
>> I send the contracts home and use assert, enforce, and unittest.
>
>Contracts are not for input validation! They are checking if the logic 
>of your program is correct or not. Think of it this way - your program 
>should behave exactly the same with or without the contracts turned on.
>
>Contracts should NOT be used for scrubbing user input, checking for 
>errors from other components, or validating any input from external to 
>the dll.
>
>If you feel the need to leave them on in a release build, then:
>1) your testing is inadequate
>2) you are using them incorrectly
>
>For example, Windows API functions check all their input. This is not 
>contract programming - it's validating user input over which Microsoft 
>has no control.

This is exactly how I look at them. However I've never tried to use
pre/post conditions. I guess it's because of the syntax.

By the way, about that image on the contracts page. Is the bullet
flying away from the D-man because it's disgusted by his extreme
ugliness? :)

Re: std.locale

[Walter Bright]

Andrei Alexandrescu wrote:
I agree. I'm having the same problem: I put a contract in there, I know 
it's as good as assert. So I can't do e.g. input validation because in 
most functions input must always be validated. I also know that 
contracts are doing the wrong thing with inheritance and can't apply to 
interfaces, which is exactly the (only?) place they'd be interesting. So 
I send the contracts home and use assert, enforce, and unittest.


Contracts are not for input validation! They are checking if the logic 
of your program is correct or not. Think of it this way - your program 
should behave exactly the same with or without the contracts turned on.


Contracts should NOT be used for scrubbing user input, checking for 
errors from other components, or validating any input from external to 
the dll.


If you feel the need to leave them on in a release build, then:
1) your testing is inadequate
2) you are using them incorrectly

For example, Windows API functions check all their input. This is not 
contract programming - it's validating user input over which Microsoft 
has no control.

Re: Null references (oh no, not again!)

[Andrei Alexandrescu]

Daniel Keep wrote:

Just noticed this hit Slashdot, and thought I might repost the abstract
here.

http://qconlondon.com/london-2009/presentation/Null+References:+The+Billion+Dollar+Mistake


I call it my billion-dollar mistake. It was the invention of the null
reference in 1965. [...] This has led to innumerable errors,
vulnerabilities, and system crashes, which have probably caused a
billion dollars of pain and damage in the last forty years. [...] More
recent programming languages like Spec# have introduced declarations
for non-null references. This is the solution, which I rejected in
1965.


  -- Sir Charles Hoare, Inventor of QuickSort, Turing Award Winner


I suggested to Walter an idea he quite took to: offering the ability of 
disabling the default constructor. This is because at root any null 
pointer was a pointer created with its default constructor. The feature 
has some interesting subtleties to it but is nothing out of the ordinary 
and the code must be written anyway for typechecking invariant 
constructors.


That, together with the up-and-coming alias this feature, will allow the 
creation of the "perfect" NonNull!(T) type constructor (along with many 
other cool things). I empathize with those who think non-null should be 
the default, but probably that won't fly with Walter.



Andrei

Re: The Sweet With

[Walter Bright]

Tomasz Sowiński wrote:

Ideas for features based on the with.

The with can make calling functions with enum arguments sexier. So instead of:
auto d = dirEntries(".", SpanMode.breadth);

you could say:
auto d = dirEntries(".", breadth);

by declaring the function as:
dirEntries(string path, with SpanMode mode);// "with" does the trick


It looks nice, but has a subtle and disastrous problem. In D, arguments 
are fully resolved *before* overloading is done. If some of the 
overloads have with declarations, then there's a nightmarish problem of 
trying to mix overloading and argument resolution together.

Re: First class lazy Interval

[Denis Koroskin]

On Tue, 03 Mar 2009 19:37:13 +0300, Tomas Lindquist Olsen 
 wrote:


On Tue, Mar 3, 2009 at 5:22 PM, Don  wrote:

Joel C. Salomon wrote:


Daniel Keep wrote:


Yes, a..b is very nice.  It's also a bad syntax for intervals.  As Don
keeps pointing out, you can't have an interval that includes int.max
with that syntax.


4 .. int.$

—Joel Salomon


I love that real.$ is the number that's even bigger than infinity .
What if the range is stored in variables?

x[a..b];
How can you set b so that it includes x[int.max]?


How about x[a..b] vs x[a..b+] ?


What would the signature of opSlice be in both case? Or different methods invoked? 

Re: Null references (oh no, not again!)

[bearophile]

Daniel Keep:
> > * Accessing arrays out-of-bounds
> > * Dereferencing null pointers
> > * Integer overflow
> > * Accessing uninitialized variables
> ...
> Note that D already solves #1 and #4, LDC could give us #3... that just
> leaves #2.  :D

Nice to see another person appreciate my desire to avoid those bugs as much as 
possible. Eventually I hope to see D avoid all four of them.
There are other bugs too, like ones derived by out-of-bounds pointer 
arithmetic, but they are less common (Cyclone and other languages are able to 
avoid them too, but it costs some).

Do you remember the ptr==null thing? As soon as D2 has full integral overflow 
checks, all people will find several bugs in their "large" D2 programs. I have 
seen this many times, switching on such overflow checks in Delphi programs :-)

Bye,
bearophile

Re: std.locale

[Daniel Keep]

Sean Kelly wrote:
> Michel Fortin wrote:
>> On 2009-03-02 14:58:26 -0500, Walter Bright
>>  said:
>>
>>> It's a silly thing, but I love the little google widget you can add
>>> to a web page to automatically translate the pages. All the D site
>>> pages have it in the left column.
>>
>> It's not a silly thing, it's hilarious. Look, Google has invented the
>> D-French language:
>>
>> -import std.stdio;
>> +std.stdio importation;
>> --
>> -delete cl;
>> +supprimer cl;
>> --
>> -s.allocated += argv.length * typeof (argv[0]).sizeof;
>> +s.allocated + = * argv.length typeof (argv [0]). sizeof;
> 
> Wow, I didn't know the standard French form for formulas was prefix
> notation :-)
> 
> 
> Sean

Wait, does this mean the French speak LISP?

No wonder I could never understand them! :O

  -- Daniel

Re: std.locale

[Sean Kelly]

Michel Fortin wrote:
On 2009-03-02 14:58:26 -0500, Walter Bright  
said:


It's a silly thing, but I love the little google widget you can add to 
a web page to automatically translate the pages. All the D site pages 
have it in the left column.


It's not a silly thing, it's hilarious. Look, Google has invented the 
D-French language:


-import std.stdio;
+std.stdio importation;
--
-delete cl;
+supprimer cl;
--
-s.allocated += argv.length * typeof (argv[0]).sizeof;
+s.allocated + = * argv.length typeof (argv [0]). sizeof;


Wow, I didn't know the standard French form for formulas was prefix 
notation :-)



Sean

Null references (oh no, not again!)

[Daniel Keep]

Just noticed this hit Slashdot, and thought I might repost the abstract
here.

http://qconlondon.com/london-2009/presentation/Null+References:+The+Billion+Dollar+Mistake

> I call it my billion-dollar mistake. It was the invention of the null
> reference in 1965. [...] This has led to innumerable errors,
> vulnerabilities, and system crashes, which have probably caused a
> billion dollars of pain and damage in the last forty years. [...] More
> recent programming languages like Spec# have introduced declarations
> for non-null references. This is the solution, which I rejected in
> 1965.

  -- Sir Charles Hoare, Inventor of QuickSort, Turing Award Winner

Serendipitous, since I just spent today trying to track down an
(expletives deleted) obscure null dereference problem.  I figure I must
be in good company if even the guy who invented null doesn't like it...

It also make me look up this old thing; it's several years old now, but
I still think it's got some good points in it.

http://www.st.cs.uni-saarland.de/edu/seminare/2005/advanced-fp/docs/sweeny.pdf

> * Accessing arrays out-of-bounds
> * Dereferencing null pointers
> * Integer overflow
> * Accessing uninitialized variables
>
> 50% of the bugs in Unreal can be traced to these problems!

Tim Sweeny isn't an amateur; he's responsible, at least in part, for one
of the most commercially successful game engines ever.  I figure if even
he has trouble with these things, it's worth trying to fix them.

Note that D already solves #1 and #4, LDC could give us #3... that just
leaves #2.  :D

  -- Daniel

Re: Recent Phobos changes

[Don]

Leandro Lucarella wrote:

Jason House, el 26 de febrero a las 12:13 me escribiste:

Don Wrote:


Jason House wrote:

There's been a flurry of commits lately. I noticed two strange things:
1. Unlike other intrinsics, bswap was not marked nothrow (only pure). This 
seems odd considering what the function does.
2. The math changes removed a lot of float-based prototypes when adding faster 
functions. That will cause difficulty for those who use floats. Ignoring speed 
loss for conversions, this will also require explicit casts.

Are these bugs, or am I missing something?
You're missing an intermediate compiler release . You're seeing stuff 
in a peculiar intermediate state. Yes, there are weird bugs.

I'm hoping that pointing out stuff when I see it is helpful because they
could get missed.  I don't know of the proper way to point out things or
if I should just stay silent on the sidelines...


Good commit messages can help a lot here. Something like "this commit is
incomplete because float stuff is missing but this will be added in the
future" could help people who like to review the code to know if missing
stuff is intentional or an error/omission.


But that's not true...
I gave the wrong impression, neither of the things which were reported 
were bugs. The float stuff has never been present on D1. There are no 
speed impacts, and no casts are required. The reported bug is false. The 
only consequence of the float stuff was if you're using 'auto' heavily, 
and even then, it's doubtful that float overloads are actually helpful.


The point I was trying to make was that (unlike all other dsource 
projects), you don't necessarily know which compiler is being used. It 
makes reviewing the code pretty difficult! Don't be alarmed.

Re: First class lazy Interval

[Tomas Lindquist Olsen]

On Tue, Mar 3, 2009 at 5:22 PM, Don  wrote:
> Joel C. Salomon wrote:
>>
>> Daniel Keep wrote:
>>>
>>> Yes, a..b is very nice.  It's also a bad syntax for intervals.  As Don
>>> keeps pointing out, you can't have an interval that includes int.max
>>> with that syntax.
>>
>> 4 .. int.$
>>
>> —Joel Salomon
>
> I love that real.$ is the number that's even bigger than infinity .
> What if the range is stored in variables?
>
> x[a..b];
> How can you set b so that it includes x[int.max]?

How about x[a..b] vs x[a..b+] ?

Re: First class lazy Interval

[Don]

Joel C. Salomon wrote:

Daniel Keep wrote:

Yes, a..b is very nice.  It's also a bad syntax for intervals.  As Don
keeps pointing out, you can't have an interval that includes int.max
with that syntax.


4 .. int.$

—Joel Salomon


I love that real.$ is the number that's even bigger than infinity .
What if the range is stored in variables?

x[a..b];
How can you set b so that it includes x[int.max]?

BTW, I'm not merely being negative, I'd love for one of these schemes to 
work.

Re: std.locale

[Sergey Gromov]

Tue, 03 Mar 2009 07:05:51 -0800, Andrei Alexandrescu wrote:

> bearophile wrote:
>> Daniel Keep:
>>> So I put contracts on everything.  Fantastic.  I do a release compile,
>>> and all that safety disappears.  So only the debug build has contracts
>>> enabled.  But it's the release build, if it crashes, that I need help
>>> diagnosing.
>> 
>> A simple solution is to not use -release for the final version of the code, 
>> but this keeps array bound controls too.
>> LDC may have already solved your problem, with extra compilation arguments 
>> that you can use to disable such controls independently from each other.
>> It's not a fault of design by contract, it's just that the D compiler 
>> switches are lumped together. It seems a simple to solve problem.
>> 
>> Bye,
>> bearophile
> 
> I agree. I'm having the same problem: I put a contract in there, I know 
> it's as good as assert. So I can't do e.g. input validation because in 
> most functions input must always be validated. I also know that 
> contracts are doing the wrong thing with inheritance and can't apply to 
> interfaces, which is exactly the (only?) place they'd be interesting. So 
> I send the contracts home and use assert, enforce, and unittest.

I'd really like to see enforce() as a built-in language feature.
assert() doesn't help in way too many situations.

Re: LLVM updates

[Frits van Bommel]

Daniel Keep wrote:


bearophile wrote:

There are some improvements in the last LLVM V.2.5, among them now LLVM 
provides intrinsics for (some) arithmetic with overflow operations:
http://llvm.org/docs/LangRef.html#int_overflow

Introducing such feature into D (LDC and more) is even simpler now :-)

Bye,
bearophile


http://icanhascheezburger.files.wordpress.com/2007/12/funny-pictures-begging-cat.jpg

DO WANT. :D


This was 5 days ago, according to dsource: 



You're welcome :).

Re: std.locale

[Andrei Alexandrescu]

bearophile wrote:

Daniel Keep:

So I put contracts on everything.  Fantastic.  I do a release compile,
and all that safety disappears.  So only the debug build has contracts
enabled.  But it's the release build, if it crashes, that I need help
diagnosing.


A simple solution is to not use -release for the final version of the code, but 
this keeps array bound controls too.
LDC may have already solved your problem, with extra compilation arguments that 
you can use to disable such controls independently from each other.
It's not a fault of design by contract, it's just that the D compiler switches 
are lumped together. It seems a simple to solve problem.

Bye,
bearophile


I agree. I'm having the same problem: I put a contract in there, I know 
it's as good as assert. So I can't do e.g. input validation because in 
most functions input must always be validated. I also know that 
contracts are doing the wrong thing with inheritance and can't apply to 
interfaces, which is exactly the (only?) place they'd be interesting. So 
I send the contracts home and use assert, enforce, and unittest.


Andrei

Re: std.locale

[bearophile]

Daniel Keep:
> So I put contracts on everything.  Fantastic.  I do a release compile,
> and all that safety disappears.  So only the debug build has contracts
> enabled.  But it's the release build, if it crashes, that I need help
> diagnosing.

A simple solution is to not use -release for the final version of the code, but 
this keeps array bound controls too.
LDC may have already solved your problem, with extra compilation arguments that 
you can use to disable such controls independently from each other.
It's not a fault of design by contract, it's just that the D compiler switches 
are lumped together. It seems a simple to solve problem.

Bye,
bearophile

Re: Actor-based Concurrency

[BLS]

hi bearophile

something similar in D :

http://www.dsource.org/projects/tango.scrapple/browser/trunk/tango/scrapple/thread

actor.d and message.d are the files you may find interesting
guess in D2 we could make it even smarter.

bjoern

bearophile wrote:

Dotnet (maybe corresponding to C# 4.x) is warming up to the ideas of 
Actor-based Concurrency, used in Erlang and Scala. It's not meant to replace, 
but to be used as an alternative in some situations (different problems 
probably require different ways to implement concurrency):

http://www.ddj.com/go-parallel/article/printableArticle.jhtml?articleID=214502187

http://weblogs.asp.net/podwysocki/archive/2009/03/02/introducing-maestro-a-dsl-for-actor-based-concurrency.aspx

http://blogs.msdn.com/maestroteam/

Bye,
bearophile

Re: const?? When and why? This is ugly!

[Andrei Alexandrescu]

Jason House wrote:

Andrei Alexandrescu Wrote:


grauzone wrote:

Sometimes, you have to leave the more spiffy features to
languages like Erlang or Haskell which are suited better for
this. For example, Erlang doesn't share any state by default,
and in Haskell, everything is immutable. But they use very
special mechanisms to make up for the problems caused by
this, and that's why you can't just pick the cherries: it
won't fit, it will feel unnatural, and everything will be a
mess. It's a good idea to copy good features from other 
languages, but never forget where you come from.

We plan to add the necessary mechanisms.

What mechanisms will these be? I'm sure others are curious too.

Still under discussion. We're considering for example
message-passing queues a la Erlang.

Andrei


Is there any chance of you/Walter posting your current conclusions
and open issues? There are many on this list, including myself, who
would enjoy participating in the design.


Most of the burden is on Bartosz.

Andrei

Re: const?? When and why? This is ugly!

[Jason House]

Andrei Alexandrescu Wrote:

> grauzone wrote:
> >>> Sometimes, you have to leave the more spiffy features to languages 
> >>> like Erlang or Haskell which are suited better for this. For example, 
> >>> Erlang doesn't share any state by default, and in Haskell, everything 
> >>> is immutable. But they use very special mechanisms to make up for the 
> >>> problems caused by this, and that's why you can't just pick the 
> >>> cherries: it won't fit, it will feel unnatural, and everything will 
> >>> be a mess. It's a good idea to copy good features from other 
> >>> languages, but never forget where you come from.
> >>
> >> We plan to add the necessary mechanisms.
> > 
> > What mechanisms will these be? I'm sure others are curious too.
> 
> Still under discussion. We're considering for example message-passing 
> queues a la Erlang.
> 
> Andrei

Is there any chance of you/Walter posting your current conclusions and open 
issues? There are many on this list, including myself, who would enjoy 
participating in the design.

Re: std.locale

[Daniel Keep]

Walter Bright wrote:
> Georg Wrede wrote:
>> We've had Walter make nice features to D that were laborious to
>> create, only to see nobody use them. It's happened, ask him.
> 
> Sure. Often the only way to see if a feature is useful is to actually
> implement it and see what happens. Some features have succeeded and
> found uses far beyond my expectations (CTFE, string mixins) while others
> have pretty much languished (design by contract, complex numbers).

I've used complex numbers before, but only when rendering fractals.
Sorry :P

As for design by contract, my problem has always been this:

Contracts let you ensure that your assumptions about program state are
never violated.  That means checking pre- and post-conditions on
functions, and invariants for classes.  Which is great.

So I put contracts on everything.  Fantastic.  I do a release compile,
and all that safety disappears.  So only the debug build has contracts
enabled.  But it's the release build, if it crashes, that I need help
diagnosing.

There's also libraries.  If you put contracts on public APIs, then your
library is only checking arguments in debug builds.  This makes release
builds faster, but also less safe.  So do you only put contracts on
internal APIs and do manual exception testing on public APIs?

I like DbC, I really do.  I just have trouble figuring out where and how
to use it properly.

  -- Daniel

Actor-based Concurrency

[bearophile]

Dotnet (maybe corresponding to C# 4.x) is warming up to the ideas of 
Actor-based Concurrency, used in Erlang and Scala. It's not meant to replace, 
but to be used as an alternative in some situations (different problems 
probably require different ways to implement concurrency):

http://www.ddj.com/go-parallel/article/printableArticle.jhtml?articleID=214502187

http://weblogs.asp.net/podwysocki/archive/2009/03/02/introducing-maestro-a-dsl-for-actor-based-concurrency.aspx

http://blogs.msdn.com/maestroteam/

Bye,
bearophile

Re: std.locale

[Gide Nwawudu]

On Mon, 02 Mar 2009 01:37:55 -0800, Walter Bright
 wrote:

>Georg Wrede wrote:
>> We've had Walter make nice features to D that were laborious to create, 
>> only to see nobody use them. It's happened, ask him.
>
>Sure. Often the only way to see if a feature is useful is to actually 
>implement it and see what happens. Some features have succeeded and 
>found uses far beyond my expectations (CTFE, string mixins) while others 
>have pretty much languished (design by contract, complex numbers).

I think DbC would be widely used if it worked with inheritance and
could possible be apply to interfaces. There is an entry in Bugzilla
and has been voted sixth up to sixth place.

http://d.puremagic.com/issues/show_bug.cgi?id=302

Gide

The Sweet With

[Tomasz Sowiński]

Ideas for features based on the with.

The with can make calling functions with enum arguments sexier. So instead of:
auto d = dirEntries(".", SpanMode.breadth);

you could say:
auto d = dirEntries(".", breadth);

by declaring the function as:
dirEntries(string path, with SpanMode mode);// "with" does the trick

At first glance, such feature seems lesser. But I noticed that for fear of 
redundancy programmers resort to cluttering the global scope with constants, or 
even worse - using bool to offer only two options. So when the API needs an 
equivalent of GUI's dropdown list, the with encourages the right way - enums.


The with can also tidy up numerous imports:

import extremely.long.package.name.module1;
import extremely.long.package.name.module2;
import extremely.long.package.name.module3;
import renamed = extremely.long.package.name.module4;
import extremely.long.package.name.module5 : selective;
...

could be compressed to:

import with (extremely.long.package.name)
{
module1;
module2;
module3;
renamed = module4;
module5 : selective;
...
}

or even:

import with (extremely.long.package.name)
module1, module2, module3, renamed = module4, module5 : selective, ... ;


What do you say?

Tomek

Re: LLVM updates

[Tomas Lindquist Olsen]

On Tue, Mar 3, 2009 at 11:58 AM, bearophile  wrote:
> There are some improvements in the last LLVM V.2.5, among them now LLVM 
> provides intrinsics for (some) arithmetic with overflow operations:
> http://llvm.org/docs/LangRef.html#int_overflow
>
> Introducing such feature into D (LDC and more) is even simpler now :-)
>
> Bye,
> bearophile
>

Frits van Bommel has already added support for the new intrinsics to LDC :D
-Tomas

Re: std.locale

[Michel Fortin]

On 2009-03-02 23:27:49 -0500, Andrei Alexandrescu 
 said:



Georg Wrede wrote:
[snip]

Well I guess what I'll do is take the path of least resistance - 
nothing. Looks like locales are rather unpopular...


Sad.

Seriously, I think if D could have locales as a standard feature it'd 
be great. Supporting various locales is often a must when you deploy an 
application, and when libraries try to do it differently you find 
yourself in a mess.


One thing I dislike in your approach is that you're designing the 
underlying data storage system before considering the API we're going 
to use. What we need the most is a standard API for localizing the 
display and input of data, and I somewhat fail to see how storing all 
the localization parameters in an Algebraic solves this problem.


I mean, let's say you want to output a localized message, perhaps we 
could do this:


writefln("Hello number %f", 123456.44);
writefln(localize("Hello number %f"), 123456.44); // default locale

Locale fr = locale("fr");
writefln(localize("Hello number %f", fr), 123456.44);

and expect this output:

Hello number 123456.44
Hello number 123,456.44
Bonjour numéro 123 456,44

?

That'd be an interesting feature. But as of yet I have no idea how 
we're supposed to use all that locale information you want to keep in 
your algebraic type; you haven't provided much examples like the one 
above where all you want is to format a string and a number. Perhaps 
it's clear in your head, but to me it's vague. Exposing all this locale 
data is useless if it isn't supported by the library's functions.


What I wrote above could work that way: localize(...) returns a 
FormattedString!(...) struct template containing a string and a 
templated function "format" for formatting its arguments. writefln 
being a templated function, it'll call toString on its first argument 
and check if it provide a "format" function, and if it does it passes 
all the other arguments through it before output.


The "localize" function could be overloaded to accept various types of 
locales. Including, but not limited to, your Algebraic locale data.


The downside of this approach is that it requires functions accepting a 
locale or a localized formatted string to be templates.



--
Michel Fortin
michel.for...@michelf.com
http://michelf.com/

Re: const?? When and why? This is ugly!

[Christopher Wright]

Andrei Alexandrescu wrote:

grauzone wrote:
Sometimes, you have to leave the more spiffy features to languages 
like Erlang or Haskell which are suited better for this. For 
example, Erlang doesn't share any state by default, and in Haskell, 
everything is immutable. But they use very special mechanisms to 
make up for the problems caused by this, and that's why you can't 
just pick the cherries: it won't fit, it will feel unnatural, and 
everything will be a mess. It's a good idea to copy good features 
from other languages, but never forget where you come from.


We plan to add the necessary mechanisms.


What mechanisms will these be? I'm sure others are curious too.


Still under discussion. We're considering for example message-passing 
queues a la Erlang.


Andrei


A library-level feature, I hope, but I'd be quite interested in seeing 
how this would be implemented as a pure function.

Re: LLVM updates

[Daniel Keep]

bearophile wrote:
> There are some improvements in the last LLVM V.2.5, among them now LLVM 
> provides intrinsics for (some) arithmetic with overflow operations:
> http://llvm.org/docs/LangRef.html#int_overflow
> 
> Introducing such feature into D (LDC and more) is even simpler now :-)
> 
> Bye,
> bearophile

http://icanhascheezburger.files.wordpress.com/2007/12/funny-pictures-begging-cat.jpg

DO WANT. :D

  -- Daniel

LLVM updates

[bearophile]

There are some improvements in the last LLVM V.2.5, among them now LLVM 
provides intrinsics for (some) arithmetic with overflow operations:
http://llvm.org/docs/LangRef.html#int_overflow

Introducing such feature into D (LDC and more) is even simpler now :-)

Bye,
bearophile