Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Tuesday, 21 May 2013 at 01:34:29 UTC, estew wrote:
But I'm not convinced it would cost us less to have NotNull!T
and Nullable!T. I feel it is cheaper to mindlessly write if(a
is null) {} when using pointers than to worry at design time
what the behaviour of a pointer should be.
As a matter of fact, most reference are never null, or are
assumed never to be null.
For instance, in DMD2.062, e2ir.c line 869 it is assumed that
irs-sclosure can't be null, when in fact it can and that lead to
an ICE (and that isn't the first one, which kind of mitigate the
strength of the arguement that this rarely happens and is easy to
fix).
Design time is the second most expensive developer time for us.
The most expensive dev. time is changing a design that turned
out to be incorrect, or is now outdated for whatever reason.
Moving pointer behaviour to be a design time issue rather than
knowing it could be NULL so check it could increase the
probability of redesign bugs creeping in.
You may not know if a reference will be nullable or not when you
write you code at first. With current model, you start writing
code as if it can't be null, and then later, when you see you in
fact need null, you now can have surprise breakage anywhere.
With a Nullable, you'll have code breakage that force you to
handle the null case. This enforce correctness instead of relying
on faith.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
I know at least two cases which T.init is commonly used.
1. Inside predicate template for type T.
template isSomething(T) {
enum isSomething = is(typeof({
//T t1;// not good if T is nested struct, or has @disable this()
//T t2 = void; auto x = t2; // not good if T is non-mutable type
T t = T.init; // avoid default construct check
...use t...
}));
}
2. Some library utilities that treats object state directly, e.g.
std.conv.emplace
Kenji Hara
2013/5/20 Maxim Fomin ma...@maxim-fomin.ru
On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:
Unfortunately this is currently not a bug.
T.init provides default initialized object image, and it *does not*
provide default constructed object. The difference is important.
That is already documented in lanugage reference.
http://dlang.org/property#init
Note: .init produces a default initialized object, not default
constructed. That means using .init is sometimes incorrect.
1. If T is a nested struct, the context pointer in T.init is null.
2. If T is a struct which has @disable this();, T.init might return a
logically incorrect object.
Kenji Hara
I think this should be fixed otherwise @disable this() is compromised.
What is rationale behind allowing .init?
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 06:10:22 UTC, Kenji Hara wrote:
I know at least two cases which T.init is commonly used.
1. Inside predicate template for type T.
template isSomething(T) {
enum isSomething = is(typeof({
//T t1;// not good if T is nested struct, or has
@disable this()
//T t2 = void; auto x = t2; // not good if T is
non-mutable type
T t = T.init; // avoid default construct check
...use t...
}));
}
2. Some library utilities that treats object state directly,
e.g.
std.conv.emplace
Kenji Hara
I see. But unfortunately this undermines @disable and defeats
arguments for using it. @disable is another feature (like ref and
@safe) which cannot be fixed be design.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 21:36:17 UTC, Andrei Alexandrescu wrote: OK, this is sensible. One question - would you be willing to type symbols as NullType!T instead of T to avoid these issues? Good question. Probably not. I think it's one of those things were the awkwardness of it not being the default would lead to lack of use (in the same way usage of pure suffers from not being default).
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 21:02:11 +0200, Idan Arye generic...@gmail.com wrote:
I don't see how Option and Maybe would have helped your bug. The problem
was that somewhere in the code the reference was perceived as null while
in fact it wasn't
What does that even mean?
- so now it will be perceived as `None`, and you will have the same
problem.
Except that now the code would be forced to handle the None case. In a way,
having nullable by default is like having a stringly typed system:
function foo( s ) {
return (s + 4) * 2; // Works great when s == 16, falls dead on its
// back when s == goobers.
}
function bar( int i ) {
return (i + 4) * 2;
}
These functions look very different. That's because they are. One of them
only takes valid parameters, the other takes any old garbage and barfs
when the wrong garbage is given to it.
Of course, if you have a string, and you want to call bar, you need to
convert the string to an int. So you end up with this:
function baz( string s ) {
return s.parseInt(
i = bar(i),
{ alert(error); });
}
Notice how the parseInt function takes two delegates? One of these
(the first) is only called when the string is valid. The other is only
called if the string is invalid. That way, we can be sure that the
failure case is handled.
Exactly the same would be the case for non-nullable pointers - if you
want to convert a nullable pointer to non-nullable, you *have* to
handle the failure case. No two ways about it.
Now, the same example with class references:
int foo(A a) {
return a.qux(); // Works great when a == new A(), falls dead on its
// back when a == null.
}
int bar(NonNull!A a) {
return a.qux();
}
See how one of these does not blow up in your face (unless you do
something stupid like create a special Nil value that will do exactly
that)? Now, for baz:
int baz(A a) {
return a.match(
(NonNull!A a) = bar(a),
(None) = -1
);
}
And this is the exact problem with nullable by default : plenty of
stuff ends up be null is some weird situation that almost never occurs
when they are assumed not to be and the program crashes.
NullPointerException now return 4 millions result on google, which is
probably around once per java developers.
This is not a problem with nullable by default - it is a problem with
implicit default values. null(or Nil, or None) are the only sane default
values for reference types - I think you would agree that having to
construct a new blank object as default value for every reference
variable would be far worse than null...
If you absolutely cannot initialize the pointer to something sensible,
then use a nullable pointer. But if non-nullable pointers are not
available, or are harder to use than nullable pointer, then people will
use nullable pointers even where non-nullable would have been a much
more fitting choice.
It does not solve the bug - it is something you HAVE to do given the
assumptions. If the reference is not nullable, and you can't set it to
it's real value until later in the code, then you have to initialize it
to some temporary value.
I don't know enough about the bug to say such things for sure, but I will
say this: If deadalnix solved the bug, he is likely in a much better
position to say anything about what would solve the problem than the rest
of us.
--
Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 20:05:02 +0200, Idan Arye generic...@gmail.com wrote: These are the assumptions I'm working with: - We can't use a nullable reference - We can't initialize the reference upon declaration to it's real value. Indeed, if that's the case, then what you're doing is fairly sensible. But if #2 is true, then #1 should never be true. -- Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 20:23:21 +0200, Walter Bright newshou...@digitalmars.com wrote: On 5/19/2013 5:02 AM, Simen Kjaeraas wrote: By definition? Pointer semantics are what we choose it to mean. Of course. But which definition is saner: For many types, it is extremely useful to have some sort of invalid value for it. null fills that role nicely for pointers, just as nan does for floating point types, and 0xFF does for UTF-8. There's not anything insane about it. The Nullable type constructor even exists in order to provide such an invalid state for types (like int) which normally do not have one. Yes, I do understand there's a role for pointers which cannot hold the invalid value. I contend that not only is there a role for them, but that most pointers should never be null. Here's two questions that convinced me: 1. How many functions that take a pointer or class reference make sense to call with null in that pointer/reference? 2. If pointers were non-nullable by default, how often would you need to reach for the nullable one? I argued in another post that nullable by default is analogous to using a string instead of an int - any number representable in an int is representable in a string, *and* the string can represent error states. But if you only want valid ints, there's no reason to use a string. -- Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
I'm surprised people still have problems with null pointers. I for one am glad D has null by default makes life easy coming from C++ and Java. I may have missed something but what happens with the following code if I could not have a null pointer? int*[] pntrs = new int*[10]; Would I need to write something like? Null!(int*)[] pntrs = new Null!(int*)[10]; Personally, I'd rather have null by default as I find it less noisy and I don't need it spelled out in the code, it is implied. No it sound like initalizing something to null, then initialize it properly, assume all over the place that it is initialized to something else, and in some rare code path it blows up. OK, so the D gurus kindly introduce for us NotNull!T, Maybe!T, Option!T and SegFault!T (just for me). Now I want to access a pointer, write code using it etc. But I need to manually track at development time whether it is NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot just have a pointer anymore, knowing it's initialised to null. Now I realise it needs to change from NotNull!T to Maybe!T...great yet more refactoring. Ok refactoring done (yay sed!) but you know what, now I need to find every access to that pointer and check for null. More error prone than this: If you are in doubt (i.e. most multi-threaded apps) then check if null, with the added comfort that D has initialised all pointers to NULL for you. If still in doubt, don't use pointers. If you want non-null pointers (please no) then it is all or nothing. Allowing some pointers null and others not, via Nullable!T or NotNull!T, immediately adds another layer of complexity. I don't want to hear: D pointers cannot be null...well ok, they can sometimes, it depends, you'll have to read the code. But don't worry, D is very easy to read... My 1 cent. Disregard if I have totally misunderstood the thread, possible as it is very late! :-) Cheers, Stewart
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 14:49:32 UTC, estew wrote: Now I want to access a pointer, write code using it etc. But I need to manually track at development time whether it is NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot just have a pointer anymore, knowing it's initialised to null. Yes and this is awesome. This is correctness enforced by type system. Because if you _don't_ track this, you have a possible error in your code. Only difference between nullable and non-nullable pointers by default is that latter _force_ you to write correct code. Which is good.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:
On 5/19/13 4:30 PM, Peter Alexander wrote:
On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:
You are blowing it out of proportion. Null references are hardly even
on the radar of the bug classes I'm encountering in the style of
programming of the three groups I worked in at Facebook, and also my
previous employers. People I meet at conferences and consulting gigs
never mention null references as a real problem, although I very often
ask about problems. I find it difficult to agree with you just to be
nice.
Just because people don't mention them as a problem doesn't mean it
isn't a problem.
For what it's worth, null pointers are a real problem in the code I
work on (games). I don't know exactly what you work on, but I find that
they are more of a problem in highly stateful, interactive
applications. Things like generic libraries, utility programs,
compilers, etc. probably won't see the same problems because they
aren't very stateful or interactive.
In my experience, null pointers are easy to fix, but the risk of them
causes people to litter their code with if (ptr) tests, often with poor
handling of the failure case, which can cause subtle bugs (no crash,
but unintended code path).
Just my 2c.
OK, this is sensible. One question - would you be willing to type
symbols as NullType!T instead of T to avoid these issues?
Thanks,
Andrei
More boiler plate code for functions that take pointers.
void foo(T)(T t)
if(isPointer!T)
{
static if(isNullable!T)
if(!t) throw
}
May also introduce then need to check for objects in the init state
(default state)
outside of NonNull in stdlib, an attribute maybe a possible answer.
void bar(@nonnull SomeClass o)
{
o = foo(); // if foo returns @nonnull, then check is not needed,
else needs a check added during assignment
}
there are a few compile time checks that can be done to prove o is not
null, if not the compiler adds runtime checks.
But really, checking for valid input is part of programming, might be
better to have lint integration that can help find these types of problems
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 8:46 PM, Steven Schveighoffer wrote: I just wanted to chime in with this understanding of the bug that I am reading from deadalnix's descriptions: SomeObj obj; shareTheObj(obj); // goes to other threads obj = new SomeObj; // initialize obj This is likely simpler than the actual problem, but I think this is the gist of it. A Non-Nullable solution WOULD solve the race: SomeObj obj; // Compiler: nope, can't do that, must initialize it. Now, with an invalid state not available like null, is the developer more likely to go through the trouble of building an invalid state for obj, in order to keep the racy behavior? No. He's just going to move the initialization: SomeObj obj = new SomeObj; shareTheObj(obj); In essence the point of the anecdote is that a Non-Nullable reference would have PROMOTED avoiding the race condition -- it would have been harder to keep the racy behavior. One can only assume the entire point was to delay initialization of the object to its first use/sharing point, so the changed pattern simply initializes the object eagerly, thus missing the benefits of the pattern. Otherwise that's an instance of the old adage initialize variables at the point of definition. That, granted, non-null references do help with. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/20/13 11:19 AM, Byron Heads wrote:
On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:
OK, this is sensible. One question - would you be willing to type
symbols as NullType!T instead of T to avoid these issues?
Thanks,
Andrei
More boiler plate code for functions that take pointers.
void foo(T)(T t)
if(isPointer!T)
{
static if(isNullable!T)
if(!t) throw
}
But this goes both ways. Regardless of the default, you'd sometimes need
to distinguish between cases. You either hurt one half of your cases or
the other half. In fact you are now showing the case that assumes
non-nullable being the default.
May also introduce then need to check for objects in the init state
(default state)
outside of NonNull in stdlib, an attribute maybe a possible answer.
void bar(@nonnull SomeClass o)
{
o = foo(); // if foo returns @nonnull, then check is not needed,
else needs a check added during assignment
}
there are a few compile time checks that can be done to prove o is not
null, if not the compiler adds runtime checks.
I think a parameterized type is a better match for non-null because it
attaches to the type, not the symbol.
But really, checking for valid input is part of programming, might be
better to have lint integration that can help find these types of problems
Agreed.
Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Mon, 20 May 2013 11:43:35 -0400, Andrei Alexandrescu wrote:
On 5/20/13 11:19 AM, Byron Heads wrote:
On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:
OK, this is sensible. One question - would you be willing to type
symbols as NullType!T instead of T to avoid these issues?
Thanks,
Andrei
More boiler plate code for functions that take pointers.
void foo(T)(T t)
if(isPointer!T)
{
static if(isNullable!T)
if(!t) throw
}
But this goes both ways. Regardless of the default, you'd sometimes need
to distinguish between cases. You either hurt one half of your cases or
the other half. In fact you are now showing the case that assumes
non-nullable being the default.
May also introduce then need to check for objects in the init state
(default state)
outside of NonNull in stdlib, an attribute maybe a possible answer.
void bar(@nonnull SomeClass o)
{
o = foo(); // if foo returns @nonnull, then check is not needed,
else needs a check added during assignment }
there are a few compile time checks that can be done to prove o is not
null, if not the compiler adds runtime checks.
I think a parameterized type is a better match for non-null because it
attaches to the type, not the symbol.
But really, checking for valid input is part of programming, might be
better to have lint integration that can help find these types of
problems
Agreed.
Andrei
What about dealing with externs you want to protect?
extern(C) void foo(@nonnull int* x);
other then that I think the library solution is fine
@nonnull Bar* b = NotNull!(NotNull!Bar) b
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 05:17:57 +0200, Walter Bright newshou...@digitalmars.com wrote: On 5/18/2013 1:39 PM, Walter Bright wrote: Already reported: http://d.puremagic.com/issues/show_bug.cgi?id=10115 And Kenji has already posted a fix! What can I say, other than Awesome! Great! One more: http://d.puremagic.com/issues/show_bug.cgi?id=1528 -- Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Mon, 20 May 2013 18:05:18 +0200, Byron Heads byron.he...@gmail.com wrote: What about dealing with externs you want to protect? extern(C) void foo(@nonnull int* x); There is nothing stopping you from declaring that with this signature: extern(C) void foo(NonNull!(int*) x); -- Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote: Unfortunately this is currently not a bug. T.init provides default initialized object image, and it *does not* provide default constructed object. The difference is important. That is already documented in lanugage reference. http://dlang.org/property#init Note: .init produces a default initialized object, not default constructed. That means using .init is sometimes incorrect. 1. If T is a nested struct, the context pointer in T.init is null. 2. If T is a struct which has @disable this();, T.init might return a logically incorrect object. Kenji Hara In that case, kindly let me understand why it is not possible to allow explicit default constructor for structs given that: 1. The constructor gets called at run time. 2. Is not considered for evaluating S.init (compile time).
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 14:57:17 UTC, Dicebot wrote:
On Monday, 20 May 2013 at 14:49:32 UTC, estew wrote:
Now I want to access a pointer, write code using it etc. But I
need to manually track at development time whether it is
NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot
just have a pointer anymore, knowing it's initialised to null.
Yes and this is awesome. This is correctness enforced by type
system. Because if you _don't_ track this, you have a possible
error in your code. Only difference between nullable and
non-nullable pointers by default is that latter _force_ you to
write correct code. Which is good.
True I grant you that, it was late when I posted :-) I've
actually come around a bit on this after sleeping on it and
rereading some of the posts. I am starting to like NotNull!T idea
but I'm a bit hesitant still with Maybe!T, Option!T.
I cannot remember the last time our team had NULL pointer issues.
We have 102 devs on four integrated products. Big enough to not
know context your code might be used in nor the implementation
details of all libraries. The only pointer troubles we see are
forget to init to NULL or reset to NULL after freeing resources.
All devs know raw pointers are initialised to NULL. We are C/C++.
So non-null pointers wouldn't make much difference to us here,
although it may make the code more readable which is always a
good thing. D needs nullable pointers though, of some form.
But I'm not convinced it would cost us less to have NotNull!T and
Nullable!T. I feel it is cheaper to mindlessly write if(a is
null) {} when using pointers than to worry at design time what
the behaviour of a pointer should be.
Design time is the second most expensive developer time for us.
The most expensive dev. time is changing a design that turned out
to be incorrect, or is now outdated for whatever reason. Moving
pointer behaviour to be a design time issue rather than knowing
it could be NULL so check it could increase the probability of
redesign bugs creeping in.
Still, I am loving the discussion in this thread it's very
interesting from both sides.
Stewart
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Saturday, May 18, 2013 22:04:08 Walter Bright wrote: On 5/18/2013 9:42 PM, Jonathan M Davis wrote: On Saturday, May 18, 2013 21:30:57 Walter Bright wrote: On 5/18/2013 9:06 PM, Jonathan M Davis wrote: The closest that there is is C++'s references, which aren't rebindable and in many ways act more like aliases than pointers. You can trivially create null references in C++: int* p = NULL; int r = *p; Yes, but they're designed with the idea that they're non-nullable. You can't assign NULL to them or check if they're NULL. It's just that it's possible to make them NULL by the trick that you just showed. I don't even think it's a trick, as it can easily happen unintentionally. Yes, but it's not something that would be done intentionally, and it's something that surprises most people. I expect that the vast majority of C++ programmers would think that it's impossible before it was explained to them. C++ references are usually sold as being non-nullable, and this is arguably a hole in their design. - Jonathan M Davis
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 06:57:16 +0200, Walter Bright
newshou...@digitalmars.com wrote:
The current solution is to rely on faith, and I remember someone
talking about
that at DConf recently.
Rely on what faith?
void foo(int* p) {} // p must never be null
void foo(NotNull!(int*) p) {}
One of these is tested at compile time, *and* includes valuable
documentation in the signature. The other is either less performant
or buggy.
Now that what other languages does is cleared, let's do some
consideration on null.
A pointer point on something. For instance, an int* point on an
integer. null
doesn't point on a integer. Non nullable pointer aren't a restricted
set of
values, as, by definition, null isn't a value that point to an int.
That doesn't
stand either.
By definition? Pointer semantics are what we choose it to mean.
Of course. But which definition is saner:
T* is either a valid pointer to a T, or a value that blows up when used
in certain ways (but not others).
or
T* is a valid pointer to T.
Of course, the latter also requires something like Maybe!T:
Maybe!T is either a valid pointer to a T, or a value on which no
operations may be performed. In order to gain access to the T, both
cases have to be handled.
--
Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 02:32:49 +0200, Jonathan M Davis jmdavisp...@gmx.com wrote: On Sunday, May 19, 2013 02:22:43 Simen Kjaeraas wrote: Or... possibly, the current holes in @disable are fixed, and NonNull!T becomes the default, because we tell people to always use them, rather than flail our arms and behave like idiots. (regular pointers are broken, use NonNull!T is a pretty good argument if it's true) I've never understood why so many people feel that nullable pointers are a problem. Clearly, many people do, but personally, I've rarely had problems with them, and there are plenty of cases where not being to make a pointer null would really suck (which is why we're forced to have std.typecons.Nullable for non-reference types). I'm not arguing against having non-nullable pointers, but I'd probably almost never use them myself, as I really don't think that they'd be buying me much. In my experince, problems with null pointers are extremely rare and easily caught. My experience is the complete opposite - I think maybe 20% of bugs at my job are caused by null references. But as you say, they are very easily fixed. That said, two things to consider: How many of the functions you write actually need to accept nullable pointers/references? If non-nullable was the default, how often would you explicitly ask for a nullable pointer/reference? -- Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 05/19/2013 06:57 AM, Walter Bright wrote: D already have thing like Nullable in the standard lib. Introducing Maybe is also pretty easy. Adding NonNullable in addition to Nullable sound like something is not quite right. Nullable is something different - it exists to give a 'null' value to things that don't have a null representation, like an int. Type Object ought not to have a null representation either. C#, for example, has a Nullable type constructor, but it still has object references that can be null. C# has no standard way to produce a non-nullable object reference. There's no non-null wrapper in Java, either. This limits those languages' static type safety as much as D's.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 12:06:01 UTC, Simen Kjaeraas wrote: My experience is the complete opposite - I think maybe 20% of bugs at my job are caused by null references. But as you say, they are very easily fixed. Sometime they are super freaking hard. I have a horror story debugging Apache Cayenne : http://cayenne.apache.org/ because it was throwing NPE on a race condition that would never show up once the code is instrumented. A reference was null for a short moment and then set to something. Due to concurrency, in extreme cases it could be seen as null where it was assumed everywhere to be set. That said, two things to consider: How many of the functions you write actually need to accept nullable pointers/references? If non-nullable was the default, how often would you explicitly ask for a nullable pointer/reference? Not that much and I'd rather be warned when it is the case.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 8:55 AM, deadalnix wrote: On Sunday, 19 May 2013 at 12:06:01 UTC, Simen Kjaeraas wrote: My experience is the complete opposite - I think maybe 20% of bugs at my job are caused by null references. But as you say, they are very easily fixed. Sometime they are super freaking hard. I have a horror story debugging Apache Cayenne : http://cayenne.apache.org/ because it was throwing NPE on a race condition that would never show up once the code is instrumented. A reference was null for a short moment and then set to something. Due to concurrency, in extreme cases it could be seen as null where it was assumed everywhere to be set. Sounds like a race problem unrelated to null. With non-null objects the race would have manifested itself in a different way, perhaps even more pernicious. Anyhow, this discussion should have finality. We could go on forever arguing the usefulness or lack thereof of non-nullable references. They didn't catch up in some languages and did in some. My personal opinion is nice to have, but not greatly compelling. It's reasonable to assume no major language changes will accommodate non-null references, so the next best thing would be to focus on a library solution. As Walter said, a library solution has the perk of allowing other interesting restricted types. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote: Sounds like a race problem unrelated to null. With non-null objects the race would have manifested itself in a different way, perhaps even more pernicious. It is both a race condition and a null problem. And having non nullable type would have been a compile time error instead of days of debugging.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 9:11 AM, deadalnix wrote: On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote: Sounds like a race problem unrelated to null. With non-null objects the race would have manifested itself in a different way, perhaps even more pernicious. It is both a race condition and a null problem. No, it's just a race condition. And having non nullable type would have been a compile time error instead of days of debugging. No, the race condition would have stayed. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote: On 5/19/13 9:11 AM, deadalnix wrote: On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote: Sounds like a race problem unrelated to null. With non-null objects the race would have manifested itself in a different way, perhaps even more pernicious. It is both a race condition and a null problem. No, it's just a race condition. And having non nullable type would have been a compile time error instead of days of debugging. No, the race condition would have stayed. Andrei I believe this claim requires an explanation: It's a good practice to initialize references(and all other types of variables) as soon as possible - and if possible, right away in the declaration. If that reference started as null, it's safe to assume it was not possible to initialized it at declaration, so it was intentionally initialized with null(if there was no initialization Java would scream at you). Now, let's assume that reference was non-nullable. It is safe to assume that this change would not remove the obstacle that prevented that reference from being initialized right away in the declaration - so you still need to initialize it to something else - let's call that something `Nil`. Nil is an object that tells you that the reference has not yet been initialized. So, in the original bug the reference could be seen as null where it was assumed everywhere to be set.. But now we don't have null - so that piece of code that thought the reference is null would now think that it is... what? The initialization value? No! Because we didn't switch from initializing the reference with null to initializing it with the later initialization value - we couldn't do it. Instead, we had to use Nil. So now, the reference 'could be seen as Nil where it was assumed everywhere to be set'... Now, if you are lucky and your Nil is the better-kind-of-null that is used in dynamic object oriented languages, you'll get a nil exception - which is just as good as null exception. But if you are not so lucky, and you had to declare Nil as a blank object of the type of that reference, you are going to have logical bug, which is far worse than exceptions...
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote: On 5/19/13 9:11 AM, deadalnix wrote: It is both a race condition and a null problem. No, it's just a race condition. And having non nullable type would have been a compile time error instead of days of debugging. No, the race condition would have stayed. That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise. You also never provided any convincing solution to the safety hole. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. I encourage you to look at this : http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare Most new languages removed nullable by default, or limited its uses (scala for instance, allow for null for limited scope). I once again want to get attention on the fact that GC change everything in regard to reference, and that the C++ situation is a bad example. Idan Arye Nil proposal make no sens in a statically typed language. And you'll find no better kind of null. We have all tools we need in D to work around null in library.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 04:57:15 UTC, Walter Bright wrote: On 5/18/2013 8:54 PM, deadalnix wrote: On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote: I understand that. But the rationale you gave for having a default constructor was to be able to disable default construction. RAII or construction based on template parameters. I know what default constructors are used for in C++. That wasn't what I asked, though. I asked for compelling rationale. I have bunch of code that goes like : auto oldVar = var; scope(exit) var = oldVar; This is begging for a RAII solution where I pass var as template parameter but would require default constructor. This is an actual problem I have right now as all save/restore are harder and harder to keep in sync for no reason and generate a lot of boilerplate. This is a problem I have right now that default constructor would solve, and this isn't the first time I hit that need.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 19:12:15 +0200, Idan Arye generic...@gmail.com wrote: It's a good practice to initialize references(and all other types of variables) as soon as possible - and if possible, right away in the declaration. If that reference started as null, it's safe to assume it was not possible to initialized it at declaration, so it was intentionally initialized with null(if there was no initialization Java would scream at you). Now, let's assume that reference was non-nullable. It is safe to assume that this change would not remove the obstacle that prevented that reference from being initialized right away in the declaration - so you still need to initialize it to something else - let's call that something `Nil`. Nil is an object that tells you that the reference has not yet been initialized. Uhm, no. Nononono. No. This is a complete and utter fallacy. What you have just done is define Nullable!(NonNullable!T). I should not have to explain too closely why this is a bad thing and should not be done. -- Simen
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 17:35:43 UTC, deadalnix wrote: Idan Arye Nil proposal make no sens in a statically typed language. And you'll find no better kind of null. We have all tools we need in D to work around null in library. That's not the point. The point is that if you couldn't initialize this reference to null, you had to initialize it to something else, so you would still get your race condition.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 17:46:13 UTC, Simen Kjaeraas wrote: On Sun, 19 May 2013 19:12:15 +0200, Idan Arye generic...@gmail.com wrote: It's a good practice to initialize references(and all other types of variables) as soon as possible - and if possible, right away in the declaration. If that reference started as null, it's safe to assume it was not possible to initialized it at declaration, so it was intentionally initialized with null(if there was no initialization Java would scream at you). Now, let's assume that reference was non-nullable. It is safe to assume that this change would not remove the obstacle that prevented that reference from being initialized right away in the declaration - so you still need to initialize it to something else - let's call that something `Nil`. Nil is an object that tells you that the reference has not yet been initialized. Uhm, no. Nononono. No. This is a complete and utter fallacy. What you have just done is define Nullable!(NonNullable!T). I should not have to explain too closely why this is a bad thing and should not be done. These are the assumptions I'm working with: - We can't use a nullable reference - We can't initialize the reference upon declaration to it's real value. The first assumption is required because we want to describe how the bug scenario deadalnix brought up would look like if references were non-nullable. The second assumption is required because if we could initialize the reference upon declaration to it's real value, we should have just done it in the first place and avoid the whole race hazard. Now, I'm not saying the solution I presented is good - I'm trying to show that given those two assumptions, we are forced to use this bad solution.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 17:51:36 UTC, Idan Arye wrote: On Sunday, 19 May 2013 at 17:35:43 UTC, deadalnix wrote: Idan Arye Nil proposal make no sens in a statically typed language. And you'll find no better kind of null. We have all tools we need in D to work around null in library. That's not the point. The point is that if you couldn't initialize this reference to null, you had to initialize it to something else, so you would still get your race condition. Note that it had to be initialized, and my patch to Cayenne was simply to reorder that initialization. The patch itself was ~5lines, and would have impossible to do with a nonnullable (except with your Nil stuff, which once again don't make any sens in a strongly type language)
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 18:05:03 UTC, Idan Arye wrote: These are the assumptions I'm working with: - We can't use a nullable reference - We can't initialize the reference upon declaration to it's real value. If you can't initialize the value, you got to assume when you use it that it may not have been initialized and handle that case. You need either an Option (where you have to be explicit about what you do when the thing is null) or a Maybe (where null is ignored and Maybe contaminate every result depending on a maybe value). The first assumption is required because we want to describe how the bug scenario deadalnix brought up would look like if references were non-nullable. The second assumption is required because if we could initialize the reference upon declaration to it's real value, we should have just done it in the first place and avoid the whole race hazard. But that is the whole point ! The damn thing should have been initialized in the first place to avoid the bug. And this should have been caught at compile time with any sane type system. And this is the exact problem with nullable by default : plenty of stuff ends up be null is some weird situation that almost never occurs when they are assumed not to be and the program crashes. NullPointerException now return 4 millions result on google, which is probably around once per java developers. Now, I'm not saying the solution I presented is good - I'm trying to show that given those two assumptions, we are forced to use this bad solution. This solution is complex, do not make any sense in a strongly typed language and don't even solve the presented case.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 5:02 AM, Simen Kjaeraas wrote: By definition? Pointer semantics are what we choose it to mean. Of course. But which definition is saner: For many types, it is extremely useful to have some sort of invalid value for it. null fills that role nicely for pointers, just as nan does for floating point types, and 0xFF does for UTF-8. There's not anything insane about it. The Nullable type constructor even exists in order to provide such an invalid state for types (like int) which normally do not have one. Yes, I do understand there's a role for pointers which cannot hold the invalid value.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 10:41 AM, deadalnix wrote: I have bunch of code that goes like : auto oldVar = var; scope(exit) var = oldVar; This is begging for a RAII solution where I pass var as template parameter but would require default constructor. This is an actual problem I have right now as all save/restore are harder and harder to keep in sync for no reason and generate a lot of boilerplate. This is a problem I have right now that default constructor would solve, and this isn't the first time I hit that need. oldVar isn't being default constructed in your example, nor can I see why you'd need a default constructor in order to use RAII for save/restore.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 18:23:22 UTC, Walter Bright wrote:
On 5/19/2013 5:02 AM, Simen Kjaeraas wrote:
By definition? Pointer semantics are what we choose it to
mean.
Of course. But which definition is saner:
For many types, it is extremely useful to have some sort of
invalid value for it. null fills that role nicely for
pointers, just as nan does for floating point types, and 0xFF
does for UTF-8.
I don't wanted to bring that up because I thought it would
confuse people, but yes, 0xFF for char is the exact same problem
and I argue in the same direction : require explicit
initialization.
There's not anything insane about it. The Nullable type
constructor even exists in order to provide such an invalid
state for types (like int) which normally do not have one.
If something can be null, you MUST do something to handle
specifically the null case. D completely fail to ensure that.
void buzz(Foo f) {
f.foo(); // Rely in faith. It is invalid and way easier to
write than the valid code, which is THE recipe for it to spread.
}
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 18:27:08 UTC, Walter Bright wrote: oldVar isn't being default constructed in your example, nor can I see why you'd need a default constructor in order to use RAII for save/restore. I need to save the value at construction and restore at destruction. I don't need any runtime parameter at construction.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 10:35 AM, deadalnix wrote: On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote: No, the race condition would have stayed. That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise. I agree with Andrei that eliminating null is not going to make a race condition into a thread safe one. You've got a serious bug that you may succeed in hiding by eliminating the null, but it's still there. From your description, I suspect the code is suffering from the double-checked locking bug (which can appear in very subtle forms). You also never provided any convincing solution to the safety hole. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. Please don't make us guess what exactly you mean by this.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 2013-05-19 19:41, deadalnix wrote:
I have bunch of code that goes like :
auto oldVar = var;
scope(exit) var = oldVar;
This is begging for a RAII solution where I pass var as template
parameter but would require default constructor. This is an actual
problem I have right now as all save/restore are harder and harder to
keep in sync for no reason and generate a lot of boilerplate.
This is a problem I have right now that default constructor would solve,
and this isn't the first time I hit that need.
You can do something like this:
void restore (alias value, alias dg) ()
{
auto tmp = value;
scope (exit)
value = tmp;
dg();
}
int a;
void foo () { a = 4 };
void main ()
{
a = 3;
restore!(a, {
foo();
});
}
The syntax isn't that pretty but it should work. I wish D had better
syntax for this, something like:
restore(a) {
foo();
}
--
/Jacob Carlborg
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 18:22:16 UTC, deadalnix wrote: On Sunday, 19 May 2013 at 18:05:03 UTC, Idan Arye wrote: These are the assumptions I'm working with: - We can't use a nullable reference - We can't initialize the reference upon declaration to it's real value. If you can't initialize the value, you got to assume when you use it that it may not have been initialized and handle that case. You need either an Option (where you have to be explicit about what you do when the thing is null) or a Maybe (where null is ignored and Maybe contaminate every result depending on a maybe value). I don't see how Option and Maybe would have helped your bug. The problem was that somewhere in the code the reference was perceived as null while in fact it wasn't - so now it will be perceived as `None`, and you will have the same problem. The first assumption is required because we want to describe how the bug scenario deadalnix brought up would look like if references were non-nullable. The second assumption is required because if we could initialize the reference upon declaration to it's real value, we should have just done it in the first place and avoid the whole race hazard. But that is the whole point ! The damn thing should have been initialized in the first place to avoid the bug. And this should have been caught at compile time with any sane type system. And this is the exact problem with nullable by default : plenty of stuff ends up be null is some weird situation that almost never occurs when they are assumed not to be and the program crashes. NullPointerException now return 4 millions result on google, which is probably around once per java developers. This is not a problem with nullable by default - it is a problem with implicit default values. null(or Nil, or None) are the only sane default values for reference types - I think you would agree that having to construct a new blank object as default value for every reference variable would be far worse than null... Now, I'm not saying the solution I presented is good - I'm trying to show that given those two assumptions, we are forced to use this bad solution. This solution is complex, do not make any sense in a strongly typed language and don't even solve the presented case. It does not solve the bug - it is something you HAVE to do given the assumptions. If the reference is not nullable, and you can't set it to it's real value until later in the code, then you have to initialize it to some temporary value.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 1:41 PM, deadalnix wrote: On Sunday, 19 May 2013 at 04:57:15 UTC, Walter Bright wrote: On 5/18/2013 8:54 PM, deadalnix wrote: On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote: I understand that. But the rationale you gave for having a default constructor was to be able to disable default construction. RAII or construction based on template parameters. I know what default constructors are used for in C++. That wasn't what I asked, though. I asked for compelling rationale. I have bunch of code that goes like : auto oldVar = var; scope(exit) var = oldVar; This is begging for a RAII solution where I pass var as template parameter but would require default constructor. No need for a default constructor. You pass the current value as a constructor parameter. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote: You also never provided any convincing solution to the safety hole. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. Please don't make us guess what exactly you mean by this. This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world. And no, putting nullcheck on access of field of sufficient offset (as propose dby Andrei) isn't enough because we have value types. Consider : S[BIG_NUMBER]* a; auto s = (*a[SLIGHTLY_BELLOW_CHECK_OFFSET]); s.fieldAccess; // May not have enough offset to trigget null check, but still can be usnafe See bug reports : http://d.puremagic.com/issues/show_bug.cgi?id=3677 http://d.puremagic.com/issues/show_bug.cgi?id=5176
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 1:12 PM, Idan Arye wrote: On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote: On 5/19/13 9:11 AM, deadalnix wrote: On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote: Sounds like a race problem unrelated to null. With non-null objects the race would have manifested itself in a different way, perhaps even more pernicious. It is both a race condition and a null problem. No, it's just a race condition. And having non nullable type would have been a compile time error instead of days of debugging. No, the race condition would have stayed. Andrei I believe this claim requires an explanation: It's a good practice to initialize references(and all other types of variables) as soon as possible - and if possible, right away in the declaration. If that reference started as null, it's safe to assume it was not possible to initialized it at declaration, so it was intentionally initialized with null(if there was no initialization Java would scream at you). Now, let's assume that reference was non-nullable. It is safe to assume that this change would not remove the obstacle that prevented that reference from being initialized right away in the declaration - so you still need to initialize it to something else - let's call that something `Nil`. Nil is an object that tells you that the reference has not yet been initialized. So, in the original bug the reference could be seen as null where it was assumed everywhere to be set.. But now we don't have null - so that piece of code that thought the reference is null would now think that it is... what? The initialization value? No! Because we didn't switch from initializing the reference with null to initializing it with the later initialization value - we couldn't do it. Instead, we had to use Nil. So now, the reference 'could be seen as Nil where it was assumed everywhere to be set'... Now, if you are lucky and your Nil is the better-kind-of-null that is used in dynamic object oriented languages, you'll get a nil exception - which is just as good as null exception. But if you are not so lucky, and you had to declare Nil as a blank object of the type of that reference, you are going to have logical bug, which is far worse than exceptions... So the race would have manifested it just the same, except under the name of Nil instead of null. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 1:35 PM, deadalnix wrote: On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote: On 5/19/13 9:11 AM, deadalnix wrote: It is both a race condition and a null problem. No, it's just a race condition. And having non nullable type would have been a compile time error instead of days of debugging. No, the race condition would have stayed. That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise. No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you. You also never provided any convincing solution to the safety hole. What's the safety hole? Objects of large static size? We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. How about using NonNull. We won't change the language at this point to make non-nullable references by default. Even you acknowledged that that's not practical. So now you contradict your own affirmation. What exactly do you sustain, and what are you asking for? I encourage you to look at this : http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare I read it. I don't buy it. Yeah, it's a point, but it's largely exaggerated for dramatic purposes. Most new languages removed nullable by default, or limited its uses (scala for instance, allow for null for limited scope). So what do you realistically think we should do, seeing that we're aiming at stability? I once again want to get attention on the fact that GC change everything in regard to reference, and that the C++ situation is a bad example. I don't understand this. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Saturday, 18 May 2013 at 20:39:29 UTC, Walter Bright wrote:
On 5/18/2013 1:22 PM, deadalnix wrote:
Many are, but I think that isn't the point we are discussing
here.
Removing all holes in @disable this will require the same
sacrifices at the ends
than default constructor would. For isntance, what should
happen in this case :
S[] ss;
ss.length = 42;
if S has @disable this ?
Already reported:
http://d.puremagic.com/issues/show_bug.cgi?id=10115
New case, will report it:
struct S
{
int a;
@disable this();
this(int) { a = 1; }
~this() { assert(a !is 0); }
alias a this;
int opCall() { return a; }
}
void main()
{
switch (S.init())
{
case 0:
assert(0); //oops
default:
}
}
By the way, here is another bug.
I think there is disagreement about @disable reliability and
usefulness and similar issues (@safe reliability too) due to
different attitude to the problem:
- As a language designer I care about whether some feature is
claimed to solve some problem - and that all, I put it on a slide
as lang advantage;
- As a programmer who writes medium importance code I care
whether the feature stops me from making bugs unintentionally. If
it does, than I consider that the feature works.
- As a programmer who writes critical code I care whether feature
prevents from problem, even made deliberately, and if it doesn't,
than the feature isn't reliable. It doesn't mean that it is
totally useless, but it does mean that its reliability
commitments are cheap.
Since in system language there is plenty of ways to deliberately
pass invalid data to the place where some validity assumptions
were made, @disable is a broken feature.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 3:10 PM, deadalnix wrote: On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote: You also never provided any convincing solution to the safety hole. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. Please don't make us guess what exactly you mean by this. This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world. Oh, the good old object of sufficient size. We know how to fix that. And no, putting nullcheck on access of field of sufficient offset (as propose dby Andrei) isn't enough because we have value types. Consider : S[BIG_NUMBER]* a; auto s = (*a[SLIGHTLY_BELLOW_CHECK_OFFSET]); s.fieldAccess; // May not have enough offset to trigget null check, but still can be usnafe See bug reports : http://d.puremagic.com/issues/show_bug.cgi?id=3677 http://d.puremagic.com/issues/show_bug.cgi?id=5176 All of the above are variations on the sufficiently large object theme. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
void buzz(Foo f) {
f.foo(); // Rely in faith. It is invalid and way easier to
write than the valid code, which is THE recipe for it to spread.
}
Shouldn't this throw a NullPointerSomething?
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote: No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you. I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame. You also never provided any convincing solution to the safety hole. What's the safety hole? Objects of large static size? Limiting object size isn't going to cut it. Or must be super restrictive (the protection is 4kb on some systems). We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. How about using NonNull. We won't change the language at this point to make non-nullable references by default. Even you acknowledged that that's not practical. So now you contradict your own affirmation. What exactly do you sustain, and what are you asking for? 1/ NonNull do not work. 2/ It isn't because it is too late to solve a problem that it magically isn't a problem anymore. Most new languages removed nullable by default, or limited its uses (scala for instance, allow for null for limited scope). So what do you realistically think we should do, seeing that we're aiming at stability? Acknowledge it was a mistake and move on. Use the analysis that need to be done to track down @disable this issue to warn about uninitialized null stuffs. I once again want to get attention on the fact that GC change everything in regard to reference, and that the C++ situation is a bad example. I don't understand this. I C or C++ you are doomed to manage reference as you need to for memory management purpose. In garbage collected languages, you ends up with way more unmanaged references, because the GC take care of them. Doing so you multiply the surface area where null bug can strike.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 19:15:47 UTC, Andrei Alexandrescu wrote: Oh, the good old object of sufficient size. We know how to fix that. And no, putting nullcheck on access of field of sufficient offset (as propose dby Andrei) isn't enough because we have value types. Consider : S[BIG_NUMBER]* a; auto s = (*a[SLIGHTLY_BELLOW_CHECK_OFFSET]); s.fieldAccess; // May not have enough offset to trigget null check, but still can be usnafe See bug reports : http://d.puremagic.com/issues/show_bug.cgi?id=3677 http://d.puremagic.com/issues/show_bug.cgi?id=5176 All of the above are variations on the sufficiently large object theme. Andrei The code above never access a field with a sufficient offset to trigger sufficiently large runtime check. Obviously, in the presented code the bug is trivial, but if the dereferences occurs across several functions, this is doomed to fail. The solutions are : prevent any conglomerate of value type to be bigger than 4kb (the protection on OSX is 4kb) or put a null check on every dereference in @safe code.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 12:13 PM, Maxim Fomin wrote: Since in system language there is plenty of ways to deliberately pass invalid data to the place where some validity assumptions were made, @disable is a broken feature. Please report all such holes to bugzilla.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 12:31 PM, Minas Mina wrote:
On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
void buzz(Foo f) {
f.foo(); // Rely in faith. It is invalid and way easier to write than the
valid code, which is THE recipe for it to spread.
}
Shouldn't this throw a NullPointerSomething?
It throws a seg fault at runtime. It *is* checked for by the hardware.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote: On 5/19/13 1:35 PM, deadalnix wrote: On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote: On 5/19/13 9:11 AM, deadalnix wrote: It is both a race condition and a null problem. No, it's just a race condition. And having non nullable type would have been a compile time error instead of days of debugging. No, the race condition would have stayed. That is ridiculous. non nullable would have made the bug non existent, and even without race condition the problem would exists. a reference is null, it container shared, then set to something else. You can put barriers all over the place to make that sequentially consistent that it wouldn't change anything and the bug would still arise. No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you. ... It is easy to buy that the buggy code would have been rejected by a stronger type system whereas the fixed code would have passed type checking. Especially given that it manifested itself as a NPE, which would not even exist.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 3:41 PM, deadalnix wrote: On Sunday, 19 May 2013 at 19:15:47 UTC, Andrei Alexandrescu wrote: Oh, the good old object of sufficient size. We know how to fix that. And no, putting nullcheck on access of field of sufficient offset (as propose dby Andrei) isn't enough because we have value types. Consider : S[BIG_NUMBER]* a; auto s = (*a[SLIGHTLY_BELLOW_CHECK_OFFSET]); s.fieldAccess; // May not have enough offset to trigget null check, but still can be usnafe See bug reports : http://d.puremagic.com/issues/show_bug.cgi?id=3677 http://d.puremagic.com/issues/show_bug.cgi?id=5176 All of the above are variations on the sufficiently large object theme. Andrei The code above never access a field with a sufficient offset to trigger sufficiently large runtime check. It does, when the pointer to the large static array is dereferenced. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 05/19/2013 09:42 PM, Walter Bright wrote:
On 5/19/2013 12:31 PM, Minas Mina wrote:
On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
void buzz(Foo f) {
f.foo(); // Rely in faith. It is invalid and way easier to write
than the
valid code, which is THE recipe for it to spread.
}
Shouldn't this throw a NullPointerSomething?
It throws a seg fault at runtime. It *is* checked for by the hardware.
Yes, but this code looks like it calls method 'foo', which is probably
its intention. Hence it is buggy.
D's current answer is the following:
void buzz(Foo f)in{assert(!!f);}body{
f.foo();
}
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 11:37 AM, deadalnix wrote: On Sunday, 19 May 2013 at 18:27:08 UTC, Walter Bright wrote: oldVar isn't being default constructed in your example, nor can I see why you'd need a default constructor in order to use RAII for save/restore. I need to save the value at construction and restore at destruction. I don't need any runtime parameter at construction. The saved value is initialized with the value to be saved. This is not default construction.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote: I encourage you to look at this : http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare I read it. I don't buy it. Yeah, it's a point, but it's largely exaggerated for dramatic purposes. (It is a talk.)
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 3:36 PM, deadalnix wrote: On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote: No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you. I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame. Your argument has been destroyed so no need to ask details about it. Replace null with invalid state and it's the same race in any system. Let's move on. What's the safety hole? Objects of large static size? Limiting object size isn't going to cut it. Or must be super restrictive (the protection is 4kb on some systems). Well you got to do what you got to do. Field accesses for objects larger than 4KB would have to be checked in @safe code. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. How about using NonNull. We won't change the language at this point to make non-nullable references by default. Even you acknowledged that that's not practical. So now you contradict your own affirmation. What exactly do you sustain, and what are you asking for? 1/ NonNull do not work. You made the argument that although it does work, people will not use it because it's not the default. That's not quite does not work. This also ruins your point because if people don't find it worth writing NonNull!T instead of T, it means non-null doesn't buy them anything worthwhile. 2/ It isn't because it is too late to solve a problem that it magically isn't a problem anymore. You are blowing it out of proportion. Null references are hardly even on the radar of the bug classes I'm encountering in the style of programming of the three groups I worked in at Facebook, and also my previous employers. People I meet at conferences and consulting gigs never mention null references as a real problem, although I very often ask about problems. I find it difficult to agree with you just to be nice. Most new languages removed nullable by default, or limited its uses (scala for instance, allow for null for limited scope). So what do you realistically think we should do, seeing that we're aiming at stability? Acknowledge it was a mistake and move on. I'd give it more thought if we designed D from scratch. I think it's safe to move on. At any rate, I'd love if we got NonNull working nicely so we accumulate more experience with it. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 19:42:59 UTC, Walter Bright wrote:
On 5/19/2013 12:31 PM, Minas Mina wrote:
On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:
void buzz(Foo f) {
f.foo(); // Rely in faith. It is invalid and way easier to
write than the
valid code, which is THE recipe for it to spread.
}
Shouldn't this throw a NullPointerSomething?
It throws a seg fault at runtime. It *is* checked for by the
hardware.
I think there is difference between catching exception and saving
data which you have typed for some period and letting harware
check the exception for you meanwile loosing your work.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 12:10 PM, deadalnix wrote: On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote: You also never provided any convincing solution to the safety hole. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. Please don't make us guess what exactly you mean by this. This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world. And we've replied to this before. But when you say give up on @safe, that implies a far more serious issue, so I want to make sure what you're talking about. I agree that we need to deal with the issue. But on a practical note, if we solve 99% of the @safe issues, and fail at 1%, that doesn't mean there is no value to @safe and we should give up on it.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 3:58 PM, Timon Gehr wrote: On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote: I encourage you to look at this : http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare I read it. I don't buy it. Yeah, it's a point, but it's largely exaggerated for dramatic purposes. (It is a talk.) (I read a transcript.) Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 4:05 PM, Walter Bright wrote: On 5/19/2013 12:10 PM, deadalnix wrote: On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote: You also never provided any convincing solution to the safety hole. We can't even add check only on some edges cases as D also have values types. The only solution we are left with that is really safe is to null check every dereference or give up on @safe. Please don't make us guess what exactly you mean by this. This isn't new and I discussed that again and again. When you dereference null, you hit the first plage, which is protected on most systems. But if you access an element with sufficient offset you bypass all protections provided by the type system and you are back in unsafe world. And we've replied to this before. But when you say give up on @safe, that implies a far more serious issue, so I want to make sure what you're talking about. I agree that we need to deal with the issue. But on a practical note, if we solve 99% of the @safe issues, and fail at 1%, that doesn't mean there is no value to @safe and we should give up on it. Almost safe == almost pregnant. @safe must be 100% safe. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote: Well you got to do what you got to do. Field accesses for objects larger than 4KB would have to be checked in @safe code. Isn't the solution as easy as doing: OR PTR:[address], 0 the same way it's done for the stack? The offset it known at compile time in most cases, so the command would be required only if both: * The object is larger than target OS' guard page size. * The position is greater than target OS' guard page size, OR is unknown at compile time.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 12:36 PM, deadalnix wrote: I C or C++ you are doomed to manage reference as you need to for memory management purpose. In garbage collected languages, you ends up with way more unmanaged references, because the GC take care of them. Doing so you multiply the surface area where null bug can strike. I still don't see the connection. I've had many more null pointer bugs in non-gc code than in gc code, but I attribute that to my being a better programmer in gc code because I did that later and was more aware of issues.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote: You are blowing it out of proportion. Null references are hardly even on the radar of the bug classes I'm encountering in the style of programming of the three groups I worked in at Facebook, and also my previous employers. People I meet at conferences and consulting gigs never mention null references as a real problem, although I very often ask about problems. I find it difficult to agree with you just to be nice. Just because people don't mention them as a problem doesn't mean it isn't a problem. For what it's worth, null pointers are a real problem in the code I work on (games). I don't know exactly what you work on, but I find that they are more of a problem in highly stateful, interactive applications. Things like generic libraries, utility programs, compilers, etc. probably won't see the same problems because they aren't very stateful or interactive. In my experience, null pointers are easy to fix, but the risk of them causes people to litter their code with if (ptr) tests, often with poor handling of the failure case, which can cause subtle bugs (no crash, but unintended code path). Just my 2c.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 1:06 PM, Andrei Alexandrescu wrote: On 5/19/13 3:58 PM, Timon Gehr wrote: (It is a talk.) (I read a transcript.) I wish more talks had transcripts available. I can read an hour talk presentation in 5 minutes.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 1:08 PM, Mr. Anonymous wrote: On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote: Well you got to do what you got to do. Field accesses for objects larger than 4KB would have to be checked in @safe code. Isn't the solution as easy as doing: OR PTR:[address], 0 the same way it's done for the stack? The offset it known at compile time in most cases, so the command would be required only if both: * The object is larger than target OS' guard page size. * The position is greater than target OS' guard page size, OR is unknown at compile time. Not a bad idea.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 1:03 PM, Maxim Fomin wrote: I think there is difference between catching exception and saving data which you have typed for some period and letting harware check the exception for you meanwile loosing your work. You can catch seg faults. It's easier on Windows, but it's doable on Linux.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 11:22 AM, deadalnix wrote: The damn thing should have been initialized in the first place to avoid the bug. Sounds like you have the double-checked locking bug. Using a different value to initialize it won't fix it. If you have a global value accessible from multiple threads, you must use synchronization. There is no way around that. If you use some other global state to check for initialization in order to avoid synchronization, you have the double checked locking bug. Yes, you do.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 2:02 PM, Walter Bright wrote: If you have a global value accessible from multiple threads, you must use synchronization. There is no way around that. If you use some other global state to check for initialization in order to avoid synchronization, you have the double checked locking bug. Yes, you do. BTW, a few years ago, I presented my clever solution to the double checked locking bug to Scott Meyers. I was very proud of it. Scott handed me my @ss (in a nice way), but I got my comeuppance. It reminded me of the hours I spent in high school determined to show that I could trisect an angle with a compass and a straightedge. There was always some tiny flaw :-)
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 4:30 PM, Peter Alexander wrote: On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote: You are blowing it out of proportion. Null references are hardly even on the radar of the bug classes I'm encountering in the style of programming of the three groups I worked in at Facebook, and also my previous employers. People I meet at conferences and consulting gigs never mention null references as a real problem, although I very often ask about problems. I find it difficult to agree with you just to be nice. Just because people don't mention them as a problem doesn't mean it isn't a problem. For what it's worth, null pointers are a real problem in the code I work on (games). I don't know exactly what you work on, but I find that they are more of a problem in highly stateful, interactive applications. Things like generic libraries, utility programs, compilers, etc. probably won't see the same problems because they aren't very stateful or interactive. In my experience, null pointers are easy to fix, but the risk of them causes people to litter their code with if (ptr) tests, often with poor handling of the failure case, which can cause subtle bugs (no crash, but unintended code path). Just my 2c. OK, this is sensible. One question - would you be willing to type symbols as NullType!T instead of T to avoid these issues? Thanks, Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote: On 5/19/13 3:36 PM, deadalnix wrote: On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote: No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you. I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame. Your argument has been destroyed so no need to ask details about it. Replace null with invalid state and it's the same race in any system. Let's move on. I don't want t to understand because I know I'm right. The fact you solved that issue and I didn't is irrelevant, I know better. You are blowing it out of proportion. Null references are hardly even on the radar of the bug classes I'm encountering in the style of programming of the three groups I worked in at Facebook, and also my previous employers. People I meet at conferences and consulting gigs never mention null references as a real problem, although I very often ask about problems. I find it difficult to agree with you just to be nice. Hiphop type annotations are non null by default. Just saying.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 21:02:57 UTC, Walter Bright wrote: On 5/19/2013 11:22 AM, deadalnix wrote: The damn thing should have been initialized in the first place to avoid the bug. Sounds like you have the double-checked locking bug. Using a different value to initialize it won't fix it. No it sound like initalizing something to null, then initialize it properly, assume all over the place that it is initialized to something else, and in some rare code path it blows up. The fact that this occurs in a multithreaded environment made it super hard to debug, but the whole thing was properly synchronized. Don't assume that I do not understand what the problem with double check locking is : http://d.puremagic.com/issues/show_bug.cgi?id=6607
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 20:30:28 UTC, Peter Alexander wrote: On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote: You are blowing it out of proportion. Null references are hardly even on the radar of the bug classes I'm encountering in the style of programming of the three groups I worked in at Facebook, and also my previous employers. People I meet at conferences and consulting gigs never mention null references as a real problem, although I very often ask about problems. I find it difficult to agree with you just to be nice. Just because people don't mention them as a problem doesn't mean it isn't a problem. For what it's worth, null pointers are a real problem in the code I work on (games). I don't know exactly what you work on, but I find that they are more of a problem in highly stateful, interactive applications. Things like generic libraries, utility programs, compilers, etc. probably won't see the same problems because they aren't very stateful or interactive. In my experience, null pointers are easy to fix, but the risk of them causes people to litter their code with if (ptr) tests, often with poor handling of the failure case, which can cause subtle bugs (no crash, but unintended code path). Just my 2c. Exactly ! Most of time null issue are easy to solve and is just the kind of crap that slow you down, and once in a while this ends up being an horrible mess. It is also true that rare path are often badly tested (and sometime you have no clue if the thing can be null or not, so you don't even know how to test it). Same argument Walter like to make about very rare failure cases apply here.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 21:36:17 UTC, Andrei Alexandrescu wrote:
On 5/19/13 4:30 PM, Peter Alexander wrot
OK, this is sensible. One question - would you be willing to
type symbols as NullType!T instead of T to avoid these issues?
Thanks,
Andrei
Trying to come up with some once-and-for-all safe way to deal
with null in languages which allow null references by default is
something I wonder about now and then. This is due to my desire
to adopt nice patterns for making my time spent working with a
language safer and easier.
Option/Maybe I think only really works when the language has
references as non-null by default. Otherwise, you're writing
verbose code for Option/Maybe and lies the rest of the time. (I'm
looking at Scala.) For D, I decided that my way to do it, to be
applied almost all of the time, is to write contracts.
void func(Klass value) in {
assert(value !is null);
} body {
}
This stops null from being passed through too many functions, so
that alleviates a lot of problems. I think contract programming
is a good solution for this, and it applies more generally to
other kinds of invalid values. This can include values other than
the default values.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 5:11 PM, Walter Bright wrote: It reminded me of the hours I spent in high school determined to show that I could trisect an angle with a compass and a straightedge. There was always some tiny flaw :-) Yah, sounds familiar. Did you prove the parallel postulate, too? Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 5:56 PM, deadalnix wrote: On Sunday, 19 May 2013 at 21:02:57 UTC, Walter Bright wrote: On 5/19/2013 11:22 AM, deadalnix wrote: The damn thing should have been initialized in the first place to avoid the bug. Sounds like you have the double-checked locking bug. Using a different value to initialize it won't fix it. No it sound like initalizing something to null, then initialize it properly, assume all over the place that it is initialized to something else, and in some rare code path it blows up. The fact that this occurs in a multithreaded environment made it super hard to debug, but the whole thing was properly synchronized. How was there a bug if everything was properly synchronized? You either describe the matter with sufficient detail, or acknowledge the destruction of your anecdote. This is going nowhere. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 5:52 PM, deadalnix wrote: On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote: On 5/19/13 3:36 PM, deadalnix wrote: On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote: No, your argument is ridiculous. You make a yarn with precious little detail that describes for everything everyone knows a textbook race condition, essentially ask that you are taking by your word that non-null would miraculously solve it, and, to add insult to injury, and when we don't buy it, you put the burden of proof on us. This is quite a trick, my hat is off to you. I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame. Your argument has been destroyed so no need to ask details about it. Replace null with invalid state and it's the same race in any system. Let's move on. I don't want t to understand because I know I'm right. The fact you solved that issue and I didn't is irrelevant, I know better. Nobody knows what the issue is. It's all unstated assumptions leading to vague claims. You have been challenged to explain it so as to count as an anecdote in favor of non-null pointers - which it may as well be! -, and failed to raise to it. Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote: How was there a bug if everything was properly synchronized? You either describe the matter with sufficient detail, or acknowledge the destruction of your anecdote. This is going nowhere. I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized. The exact same problem arise quite often in the single threaded world : a reference is set to null, the dev try to be clever when initializing it, in a rare case it isn't, and everything blows up when it occurs. It is simply easier to reproduce when things are single threaded, and are often quickly debugged in that case. As explained, the multithreaded environment makes it super hard to debug, not the primary cause of the issue. The simply consistent in moving the initialization where it was set to null in the first place. It is an instance of the very classic something may be null and code use it assuming it is never null.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 3:30 PM, Andrei Alexandrescu wrote: On 5/19/13 5:11 PM, Walter Bright wrote: It reminded me of the hours I spent in high school determined to show that I could trisect an angle with a compass and a straightedge. There was always some tiny flaw :-) Yah, sounds familiar. Did you prove the parallel postulate, too? No. That one was intuitively obvious!
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 4:06 PM, deadalnix wrote: On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote: How was there a bug if everything was properly synchronized? You either describe the matter with sufficient detail, or acknowledge the destruction of your anecdote. This is going nowhere. I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized. so, you have: == Thread A Thread B lock p = null unlock lock *p = ... unlock lock p = new ... unlock == Although you are using locks, it still isn't properly synchronized. Changing the p=null to p=someothernonnullvalue will not fix it.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/13 7:06 PM, deadalnix wrote: On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote: How was there a bug if everything was properly synchronized? You either describe the matter with sufficient detail, or acknowledge the destruction of your anecdote. This is going nowhere. I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized. How does another thread thread accesses the object owning the lock when the assignment occurs under lock? How would non-null fix this? Would the object have type Maybe? Andrei
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, May 19, 2013 09:08:53 Andrei Alexandrescu wrote: Anyhow, this discussion should have finality. We could go on forever arguing the usefulness or lack thereof of non-nullable references. They didn't catch up in some languages and did in some. My personal opinion is nice to have, but not greatly compelling. It's reasonable to assume no major language changes will accommodate non-null references, so the next best thing would be to focus on a library solution. As Walter said, a library solution has the perk of allowing other interesting restricted types. Wow, this thread really expanded since I looked at it last night. Yeah, I thought that it was clear that NonNullable or NotNull or whatever we want to call it was going to go in std.typecons and that that was going to be our solution to non-nullable references. This is one of those discussions that I should probably just stay out of, since it never seems to go anywhere useful. We've already decided on a solution. We just haven't gotten it into the standard library yet. - Jonathan M Davis
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 3:04 PM, deadalnix wrote: Same argument Walter like to make about very rare failure cases apply here. 1. rare as in programmers rarely create such a bug 2. rare as in being rare for an existing bug to show itself I was referring to (1), while you are referring to (2).
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 23:07:00 UTC, deadalnix wrote: On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote: How was there a bug if everything was properly synchronized? You either describe the matter with sufficient detail, or acknowledge the destruction of your anecdote. This is going nowhere. I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized. The exact same problem arise quite often in the single threaded world : a reference is set to null, the dev try to be clever when initializing it, in a rare case it isn't, and everything blows up when it occurs. It is simply easier to reproduce when things are single threaded, and are often quickly debugged in that case. As explained, the multithreaded environment makes it super hard to debug, not the primary cause of the issue. The simply consistent in moving the initialization where it was set to null in the first place. It is an instance of the very classic something may be null and code use it assuming it is never null. So this is not a problem of nullableness - rather this is a problem of mutability.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote: On 5/19/2013 1:03 PM, Maxim Fomin wrote: I think there is difference between catching exception and saving data which you have typed for some period and letting harware check the exception for you meanwile loosing your work. You can catch seg faults. It's easier on Windows, but it's doable on Linux. What's the rational for not doing this by default in D? Wouldn't a MemoryAccessError or similar be better than crashing out with SIGSEGV ? I have no idea about the consequences of this (other than tempting people to catch a segfault when they shouldn't, which is pretty much always).
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sunday, 19 May 2013 at 23:29:53 UTC, Walter Bright wrote: On 5/19/2013 4:06 PM, deadalnix wrote: On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote: How was there a bug if everything was properly synchronized? You either describe the matter with sufficient detail, or acknowledge the destruction of your anecdote. This is going nowhere. I explained over and over. A field is initialized to null, while the object lock is owned, and later to its value, while it is locked. In the meantime, another thread access the object, owning the lock, assuming the field is always initialized. so, you have: == Thread A Thread B lock p = null Here p = null is implicit, this is part of the fun. The initialisation is still properly synchronized. unlock lock *p = ... It was in java, so more somethign like p.foo(); But yes. unlock lock p = new ... unlock == Although you are using locks, it still isn't properly synchronized. Changing the p=null to p=someothernonnullvalue will not fix it. No race condition exists in that program. The error lie in improper initialization of p in the first place, which should never has been null. The example looks dumb as this, you have to imagine the pattern hidden in thousands of LOC. The code is bugguy, but you'll find no undefined threading effect? What happen is perfectly defined here and no thread access shared data without owning the lock.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 00:23:59 UTC, John Colvin wrote: On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote: On 5/19/2013 1:03 PM, Maxim Fomin wrote: I think there is difference between catching exception and saving data which you have typed for some period and letting harware check the exception for you meanwile loosing your work. You can catch seg faults. It's easier on Windows, but it's doable on Linux. What's the rational for not doing this by default in D? Wouldn't a MemoryAccessError or similar be better than crashing out with SIGSEGV ? I have no idea about the consequences of this (other than tempting people to catch a segfault when they shouldn't, which is pretty much always). https://github.com/D-Programming-Language/druntime/blob/master/src/etc/linux/memoryerror.d Still you can have weird effect when the code above throw in a middle of a C routine or something. As C don't know about D exception, it is definitively something to be aware of.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 00:09:23 UTC, Walter Bright wrote: On 5/19/2013 3:04 PM, deadalnix wrote: Same argument Walter like to make about very rare failure cases apply here. 1. rare as in programmers rarely create such a bug 2. rare as in being rare for an existing bug to show itself I was referring to (1), while you are referring to (2). When you talk about UNIX utilities not handling properly a full filesystem for instance, you are referring to 1.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Sun, 19 May 2013 16:03:24 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 5/19/13 3:36 PM, deadalnix wrote: I described a very usual null bug : something is set to null, then to a specific value. It is assumed not to be null. In a specific case it is null and everything explode. The concurrent context here made it especially hard to debug, but isn't the cause of the bug. Additionally, if you don't have enough information to understand what I'm saying, you are perfectly allowed to ask for additional details This isn't a shame. Your argument has been destroyed so no need to ask details about it. Replace null with invalid state and it's the same race in any system. Let's move on. I just wanted to chime in with this understanding of the bug that I am reading from deadalnix's descriptions: SomeObj obj; shareTheObj(obj); // goes to other threads obj = new SomeObj; // initialize obj This is likely simpler than the actual problem, but I think this is the gist of it. A Non-Nullable solution WOULD solve the race: SomeObj obj; // Compiler: nope, can't do that, must initialize it. Now, with an invalid state not available like null, is the developer more likely to go through the trouble of building an invalid state for obj, in order to keep the racy behavior? No. He's just going to move the initialization: SomeObj obj = new SomeObj; shareTheObj(obj); In essence the point of the anecdote is that a Non-Nullable reference would have PROMOTED avoiding the race condition -- it would have been harder to keep the racy behavior. I'm not saying that I think we need NN references as a compiler-supported type, or that it needs to be the default, or that NN references ALWAYS solve race conditions. I'm just pointing out what I see is an obvious misinterpretation of the underlying story. -Steve
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 05/20/2013 01:39 AM, Andrei Alexandrescu wrote:
On 5/19/13 7:06 PM, deadalnix wrote:
On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:
How was there a bug if everything was properly synchronized? You
either describe the matter with sufficient detail, or acknowledge the
destruction of your anecdote. This is going nowhere.
I explained over and over. A field is initialized to null, while the
object lock is owned, and later to its value, while it is locked. In the
meantime, another thread access the object, owning the lock, assuming
the field is always initialized.
How does another thread thread accesses the object owning the lock
when the assignment occurs under lock?
lock{ initialize to null. }
lock{ in the meantime assume correctly initialized. }
lock{ initialize correctly. }
This is nothing new. I think he has been pretty clear about what the
issue is from the beginning.
How would non-null fix this? Would the object have type Maybe?
This is one possibility. In this case, the type system would have
prevented the null dereference.
In the other case, the type system would have caught the invalid
initialization.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 05/20/2013 02:33 AM, deadalnix wrote: On Monday, 20 May 2013 at 00:09:23 UTC, Walter Bright wrote: On 5/19/2013 3:04 PM, deadalnix wrote: Same argument Walter like to make about very rare failure cases apply here. 1. rare as in programmers rarely create such a bug 2. rare as in being rare for an existing bug to show itself I was referring to (1), while you are referring to (2). When you talk about UNIX utilities not handling properly a full filesystem for instance, you are referring to 1. You mean 2.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
Unfortunately this is currently not a bug.
T.init provides default initialized object image, and it *does not*
provide default constructed object. The difference is important.
That is already documented in lanugage reference.
http://dlang.org/property#init
Note: .init produces a default initialized object, not default
constructed. That means using .init is sometimes incorrect.
1. If T is a nested struct, the context pointer in T.init is null.
2. If T is a struct which has @disable this();, T.init might return a
logically incorrect object.
Kenji Hara
2013/5/20 Maxim Fomin ma...@maxim-fomin.ru
On Saturday, 18 May 2013 at 20:39:29 UTC, Walter Bright wrote:
On 5/18/2013 1:22 PM, deadalnix wrote:
Many are, but I think that isn't the point we are discussing here.
Removing all holes in @disable this will require the same sacrifices at
the ends
than default constructor would. For isntance, what should happen in this
case :
S[] ss;
ss.length = 42;
if S has @disable this ?
Already reported:
http://d.puremagic.com/issues/**show_bug.cgi?id=10115http://d.puremagic.com/issues/show_bug.cgi?id=10115
New case, will report it:
struct S
{
int a;
@disable this();
this(int) { a = 1; }
~this() { assert(a !is 0); }
alias a this;
int opCall() { return a; }
}
void main()
{
switch (S.init())
{
case 0:
assert(0); //oops
default:
}
}
By the way, here is another bug.
I think there is disagreement about @disable reliability and usefulness
and similar issues (@safe reliability too) due to different attitude to
the problem:
- As a language designer I care about whether some feature is claimed to
solve some problem - and that all, I put it on a slide as lang advantage;
- As a programmer who writes medium importance code I care whether the
feature stops me from making bugs unintentionally. If it does, than I
consider that the feature works.
- As a programmer who writes critical code I care whether feature prevents
from problem, even made deliberately, and if it doesn't, than the feature
isn't reliable. It doesn't mean that it is totally useless, but it does
mean that its reliability commitments are cheap.
Since in system language there is plenty of ways to deliberately pass
invalid data to the place where some validity assumptions were made,
@disable is a broken feature.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 5:28 PM, deadalnix wrote: The error lie in improper initialization of p in the first place, which should never has been null. The example looks dumb as this, you have to imagine the pattern hidden in thousands of LOC. I would find a design that declared a variable in one place, then initialized it in another, while releasing the lock in between as a bad design pattern to begin with. What other default initialized types could be there? What about an int default initialized to 0, yet code in another thread expects it to be some other value? I suspect there'd be a lot more bugs in it than just null pointer initializations. It might be time to engineer a new pattern so you don't have to inspect thousands of LOC to manually verify correctness.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/19/2013 5:23 PM, John Colvin wrote: On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote: On 5/19/2013 1:03 PM, Maxim Fomin wrote: I think there is difference between catching exception and saving data which you have typed for some period and letting harware check the exception for you meanwile loosing your work. You can catch seg faults. It's easier on Windows, but it's doable on Linux. What's the rational for not doing this by default in D? Wouldn't a MemoryAccessError or similar be better than crashing out with SIGSEGV ? Writing a seg fault handler under Linux has a large number of weird constraints. I have no idea about the consequences of this (other than tempting people to catch a segfault when they shouldn't, which is pretty much always). At some point, all the scaffolding and workarounds to try to prevent programmers from having to deal with the underlying reality of how the system works is not appropriate for a systems programming language.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 01:08:16 UTC, Walter Bright wrote: On 5/19/2013 5:28 PM, deadalnix wrote: The error lie in improper initialization of p in the first place, which should never has been null. The example looks dumb as this, you have to imagine the pattern hidden in thousands of LOC. I would find a design that declared a variable in one place, then initialized it in another, while releasing the lock in between as a bad design pattern to begin with. I cannot agree more. This is what made tracking the cause of the bug super hard. What other default initialized types could be there? What about an int default initialized to 0, yet code in another thread expects it to be some other value? I suspect there'd be a lot more bugs in it than just null pointer initializations. It might be time to engineer a new pattern so you don't have to inspect thousands of LOC to manually verify correctness. I didn't programed Apache Cayenne, int he first place. But I had to patch it anyway.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote: Unfortunately this is currently not a bug. T.init provides default initialized object image, and it *does not* provide default constructed object. The difference is important. That is already documented in lanugage reference. http://dlang.org/property#init Note: .init produces a default initialized object, not default constructed. That means using .init is sometimes incorrect. 1. If T is a nested struct, the context pointer in T.init is null. 2. If T is a struct which has @disable this();, T.init might return a logically incorrect object. Kenji Hara I think this should be fixed otherwise @disable this() is compromised. What is rationale behind allowing .init?
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Saturday, 18 May 2013 at 05:19:30 UTC, Maxim Fomin wrote: On Saturday, 18 May 2013 at 02:34:46 UTC, Andrej Mitrovic wrote: On 5/18/13, Timothee Cour thelastmamm...@gmail.com wrote: S.init should be known at compile time by the spec. But why is this needed? It is needed in any place where such object is created due to default initialization. However if default ctor is CTFEable, dmd can support this feature. Another solution is to issue an error when such object isn't initialized explicitely. This isn't very difficult to do.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/17/13, Walter Bright wal...@digitalmars.com wrote: I oppose this. D has a lot of nice features because of the .init property. Default constructors wreck that. So much great feature like : - null all over the place. - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to ensure that it if effectively non nullable. - Having to check for .init state all over the place (which have a runtime cost, in addition to be error prone). RefCounted implementation is emblematic of that.
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On 5/18/2013 12:08 AM, deadalnix wrote: On 5/17/13, Walter Bright wal...@digitalmars.com wrote: I oppose this. D has a lot of nice features because of the .init property. Default constructors wreck that. So much great feature like : - null all over the place. - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to ensure that it if effectively non nullable. - Having to check for .init state all over the place (which have a runtime cost, in addition to be error prone). RefCounted implementation is emblematic of that. What default would you use for non-null pointers?
Re: Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)
On Saturday, May 18, 2013 01:46:16 Igor Stepanov wrote: When programmer see constructor, he think that it will be evaluated each time when he create a new object. This code will seem obvious for any C++ coder. And this coder will be deceived. If we'll add default constructor to structs, we'll must safe of illusion, that constructor is called any time when new object have been created. And reject all cases when we can't provide this illusion. Also, I really don't think that there's much point in having a default constructor if it's not executed at runtime. If it's going to be known at compile time, then just default initialize all of the members for the init value. What would a default constructor buy you? It seems to me that a default constructor is useful in cases where you want to do stuff at runtime and the result can differ between executions of the program and possibly between instantions of that object - e.g. grabbing the computer's current IP or setting up the hourglass to display (which is how the hourglass is done in MFC). And given the advantages of knowing a type's default value at compile time, I just don't see how we can have it be determined at runtime. So, while I think that there are cases where a default constructor could be quite useful, I don't see how we could usefully add it to D given how D deals with default initialization and everything that it does with that. - Jonathan M Davis
