Re: T[new] misgivings
On Fri, 16 Oct 2009 05:49:12 -0400, Walter Bright newshou...@digitalmars.com wrote: Don wrote: There are two sensible options: I see the question as, is T[new] a value type or a reference type? I see it as a reference type, and so assignment should act like a reference assignment, not a value assignment. Andrei says you think arrays are like slices with some extra functionality, but slices are *not* a reference type, they are hybrids. Do you think T[new] arrays should be fully reference types? (I do) Otherwise, if you keep the length is a value type semantic, you get the same crappy appending behavior we have now. -Steve
Re: T[new] misgivings
Steven Schveighoffer wrote: On Fri, 16 Oct 2009 05:49:12 -0400, Walter Bright newshou...@digitalmars.com wrote: Don wrote: There are two sensible options: I see the question as, is T[new] a value type or a reference type? I see it as a reference type, and so assignment should act like a reference assignment, not a value assignment. Andrei says you think arrays are like slices with some extra functionality, but slices are *not* a reference type, they are hybrids. Do you think T[new] arrays should be fully reference types? (I do) I might have misrepresented his position. We both think T[new] is a reference type, and it was implemented that way in the now defunct feature. Andrei
Re: T[new] misgivings
Andrei Alexandrescu Wrote: Paradoxically this seems to be conducive to subtle efficiency issues. For example, consider: int[new] a; ... a = [1, 2, 3]; What should that do? a = new Appender!int([1,2,3]); What you describe is more like StringBuilder, and, yes, things like that require full-blown API.
Re: T[new] misgivings
Thu, 15 Oct 2009 21:55:07 -0500, Andrei Alexandrescu wrote: int[new] a; ... a = [1, 2, 3]; What should that do? To me a is an array, a reference type. Therefore assignment here means rebinding a to a new array created from a literal. A: Ok, then how do I say the common operation I want to overwrite whatever the array had with 1, 2, 3? I can only presume there must be an obvious and simple way to do so, and I thought a = [1, 2, 3] was the obvious syntax to achieve that. I'd say a.replace([1, 2, 3]); T[new] a; T[] b; ... a = b; What should that do? Error: type mismatch. Use a = b.dup;
Re: T[new] misgivings
Thu, 15 Oct 2009 23:18:22 -0500, Andrei Alexandrescu wrote: Jeremie Pelletier wrote: Andrei Alexandrescu wrote: Jeremie Pelletier wrote: Andrei Alexandrescu wrote: Walter Bright wrote: Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new] Then your argument building on similarity between the two is weakened. T[new] a; T[] b; ... a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Andrei Simple, assignment to a fails 'cannot cast T[3] to T[new]'. It's already consistent with slices of different types: char[] a = foo; // error, cannot cast immutable(char)[] to char[] int[new] b = [1, 2, 3]; // error, cannot cast int[3] to int[new] you have to do: char[] a = foo.dup; int[new] b = [1, 2, 3].dup; Jeremie I'd be _very_ unhappy to have to explain to people how in the world we managed to make the most intuitive syntax not work at all. Andrei I agree it can be confusing, the first time i tried to assign a string literal to a char[] in D2 I had to pause for a second to understand what was happening :) But what I don't like is that assigning memory from the static data segment to a resizable array isn't safe. Unless the GC can detect that the memory it is trying to resize isn't part of the heap and automatically create a new allocation for it, you're gonna have nasty side effects. The compiler could also implicitly copy the slice, but then it should also automatically copy a foo literal when assigned to char[] to keep consistent. Jeremie Speaking of which, a funny fallout of this a = [1, 2, 3] thing is that we, while striving to avoid all hidden allocations, ended up doing the worst hidden allocation with the simplest and most intuitive syntax. How about this specification: [1, 2, 3] is an array literal which *allocates* an array but the allocation can be optimized away if assigned to a slice.
Re: T[new] misgivings
Just to understand it: int[new] a; int[new] b; a = [1,2,3]; b = a; In your book, the last statement would copy contents of a into b and b.ptr != a.ptr while according to walter, b would rebind to a?
Re: T[new] misgivings
Andrei Alexandrescu wrote: I talked to Walter about T[new] today and it seems we are having a disagreement. The problem is that I believe T[new] is a container, whereas Walter believes T[new] is nothing but a slice with a couple of extra operations. Paradoxically this seems to be conducive to subtle efficiency issues. For example, consider: int[new] a; ... a = [1, 2, 3]; What should that do? If we made array literals immutable, it'd be obvious. There are two sensible options: (1) An error. a = [1, 2, 3].dup; should have the semantics Walter describes. (2) Be equivalent to a.length = 3; a[] = [1,2,3]; (Andrei semantics) But in case (2), char[new] x = abc; should also compile (without a .dup). But I don't understand how the whole thing works. int[new] a = [1,2,3,4].dup; int[] b = a[0..3]; a.length = 1; int c = b[2]; How can this be legal in Safe D ? Without reference counting, the only option I can think of is to make it illegal to reduce the length of a T[new] array: you need to reallocate if you want to shrink it.
Re: T[new] misgivings
Max Samukha wrote:
On Thu, 15 Oct 2009 21:55:07 -0500, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
I talked to Walter about T[new] today and it seems we are having a
disagreement.
I'd prefer Walter's way with a provision that array literals are
immutable and allocated statically:
immutable(int)[] a = [1, 2, 3]; // no allocation here
int[new] b = [1, 2]; // new storage is allocated and the literal is
copied there
int[] a = [1, 2, 3]; // error. dup needed
auto c = [1, 2, 3]; // c is of type immutable(int)[]
b[] = c; //b's length is changed and c's contents copied to b's
storage
auto d = [1, 2, 3].dup; // d is of type int[new]
auto e = [1, 2, 3].idup; // e is of type immutable(int)[new]
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
Re: T[new] misgivings
Don wrote: There are two sensible options: I see the question as, is T[new] a value type or a reference type? I see it as a reference type, and so assignment should act like a reference assignment, not a value assignment.
Re: T[new] misgivings
Don wrote:
Max Samukha wrote:
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
Right. Assignment of a reference type does not copy the values of what
is referred to. Only the reference is copied.
I think it would be very strange to have T[] behave like a reference
type (which it does now) and T[new] to behave like a value type.
Re: T[new] misgivings
On Fri, 16 Oct 2009 02:53:20 -0700, Walter Bright
newshou...@digitalmars.com wrote:
Don wrote:
Max Samukha wrote:
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
Right. Assignment of a reference type does not copy the values of what
is referred to. Only the reference is copied.
I think it would be very strange to have T[] behave like a reference
type (which it does now) and T[new] to behave like a value type.
By true reference type, I meant:
struct ArrayRef
{
Array* ptr;
}
struct Array
{
void* data;
size_t length;
// size_t capacity;
}
The fat reference approach for arrays has sucked from day one (IMHO).
I think performance-critical code will use slices (read - ranges)
anyway, so the additional allocation and indirection is not a big
issue.
Re: T[new] misgivings
On 2009-10-16 11:49:12 +0200, Walter Bright newshou...@digitalmars.com said: Don wrote: There are two sensible options: I see the question as, is T[new] a value type or a reference type? I see it as a reference type, and so assignment should act like a reference assignment, not a value assignment. I also see T[new] as a reference type. Unfortunately this precludes some optimizations like putting size and capacity directly at the start of the allocated array, avoiding a redirection to access data: The caller object would miss the changes if the block is reallocated. T[new] a; T[new] b; b.length=LongLength; a.length=? But as putting them at the start of the array might have an adverse effect on optimizations that expect special alignment (vector operations) maybe not everything is bad. Fawzi
Re: T[new] misgivings
On 2009-10-16 13:54:03 +0200, Max Samukha spam...@d-coding.com said:
On Fri, 16 Oct 2009 02:53:20 -0700, Walter Bright
newshou...@digitalmars.com wrote:
Don wrote:
Max Samukha wrote:
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
Right. Assignment of a reference type does not copy the values of what
is referred to. Only the reference is copied.
I think it would be very strange to have T[] behave like a reference
type (which it does now) and T[new] to behave like a value type.
By true reference type, I meant:
struct ArrayRef
{
Array* ptr;
}
struct Array
{
void* data;
size_t length;
// size_t capacity;
}
The fat reference approach for arrays has sucked from day one (IMHO).
I think performance-critical code will use slices (read - ranges)
anyway, so the additional allocation and indirection is not a big
issue.
Yes exactly his is the most logically pleasing handling of resizable
array, a resizable array is a ArrayRef (and the capacity there should
not be commented out).
Obviously this has a cost (extra indirection to access data).
Fawzi
Re: T[new] misgivings
Fawzi Mohamed wrote:
On 2009-10-16 13:54:03 +0200, Max Samukha spam...@d-coding.com said:
On Fri, 16 Oct 2009 02:53:20 -0700, Walter Bright
newshou...@digitalmars.com wrote:
Don wrote:
Max Samukha wrote:
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that
block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
Right. Assignment of a reference type does not copy the values of what
is referred to. Only the reference is copied.
I think it would be very strange to have T[] behave like a reference
type (which it does now) and T[new] to behave like a value type.
By true reference type, I meant:
struct ArrayRef
{
Array* ptr;
}
struct Array
{
void* data;
size_t length;
// size_t capacity;
}
The fat reference approach for arrays has sucked from day one (IMHO).
I think performance-critical code will use slices (read - ranges)
anyway, so the additional allocation and indirection is not a big
issue.
Yes exactly his is the most logically pleasing handling of resizable
array, a resizable array is a ArrayRef (and the capacity there should
not be commented out).
Obviously this has a cost (extra indirection to access data).
Fawzi
Yes, but you could allocate the data immediately after the Array
structure, so you only have one allocation. And in the common case,
where it never exceeds the original capacity, they stay together and
preserve cache locality.
void *data; // = raw_data;
size_t length;
size_t capacity; // = 512
ubyte[512] raw_data;
Re: T[new] misgivings
On Fri, 16 Oct 2009 14:25:44 +0200, Don nos...@nospam.com wrote: Yes, but you could allocate the data immediately after the Array structure, so you only have one allocation. And in the common case, where it never exceeds the original capacity, they stay together and preserve cache locality. void *data; // = raw_data; size_t length; size_t capacity; // = 512 ubyte[512] raw_data; Great! And if the length later exceeds the capacity, try to reallocate in place. If impossible, allocate a new block, copy the data, adjust the data pointer and shrink the original block to the size of the Array struct. Right?
Re: T[new] misgivings
Don wrote:
Max Samukha wrote:
On Thu, 15 Oct 2009 21:55:07 -0500, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
I talked to Walter about T[new] today and it seems we are having a
disagreement.
I'd prefer Walter's way with a provision that array literals are
immutable and allocated statically:
immutable(int)[] a = [1, 2, 3]; // no allocation here
int[new] b = [1, 2]; // new storage is allocated and the literal is
copied there
int[] a = [1, 2, 3]; // error. dup needed
auto c = [1, 2, 3]; // c is of type immutable(int)[]
b[] = c; //b's length is changed and c's contents copied to b's
storage
auto d = [1, 2, 3].dup; // d is of type int[new]
auto e = [1, 2, 3].idup; // e is of type immutable(int)[new]
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
It makes sense to make array literals immutable. The trouble is you
won't be able to create arrays of most interesting types that way.
class Widget { ... }
Widget w1, w2, w3;
auto arr = [ w1, w2, w3 ]; // error!
Walter brought up the same argument at some point. He compared array
literals with string literals. No! String literals only contain
statically-known characters. Array literals may contain anything.
Andrei
Re: T[new] misgivings
Walter Bright wrote: Don wrote: There are two sensible options: I see the question as, is T[new] a value type or a reference type? I see it as a reference type, and so assignment should act like a reference assignment, not a value assignment. I understand that, but to me that's an example of the good intentions that pave the way to hell. All of a sudden we have the best syntax there is either being surreptitiously inefficient, or not work at all. Why not see arrays as what they really are? They are a struct with a pointer inside it. The struct has opAssign. Period. Why see the arrays in a way that's ungainful? Andrei
Re: T[new] misgivings
Walter Bright wrote: I think it would be very strange to have T[] behave like a reference type (which it does now) and T[new] to behave like a value type. T[] is not a reference type. Andrei
Re: T[new] misgivings
Max Samukha wrote: On Fri, 16 Oct 2009 14:25:44 +0200, Don nos...@nospam.com wrote: Yes, but you could allocate the data immediately after the Array structure, so you only have one allocation. And in the common case, where it never exceeds the original capacity, they stay together and preserve cache locality. void *data; // = raw_data; size_t length; size_t capacity; // = 512 ubyte[512] raw_data; Great! And if the length later exceeds the capacity, try to reallocate in place. If impossible, allocate a new block, copy the data, adjust the data pointer and shrink the original block to the size of the Array struct. Right? That's the idea. The only problem that Walter pointed out was that moving GCs may have a problem with internal pointers. Andrei
Re: T[new] misgivings
Don wrote:
Andrei Alexandrescu wrote:
Don wrote:
Max Samukha wrote:
On Thu, 15 Oct 2009 21:55:07 -0500, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
I talked to Walter about T[new] today and it seems we are having a
disagreement.
I'd prefer Walter's way with a provision that array literals are
immutable and allocated statically:
immutable(int)[] a = [1, 2, 3]; // no allocation here
int[new] b = [1, 2]; // new storage is allocated and the literal is
copied there
int[] a = [1, 2, 3]; // error. dup needed
auto c = [1, 2, 3]; // c is of type immutable(int)[]
b[] = c; //b's length is changed and c's contents copied to b's
storage
auto d = [1, 2, 3].dup; // d is of type int[new]
auto e = [1, 2, 3].idup; // e is of type immutable(int)[new]
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that
block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
It makes sense to make array literals immutable. The trouble is you
won't be able to create arrays of most interesting types that way.
class Widget { ... }
Widget w1, w2, w3;
auto arr = [ w1, w2, w3 ]; // error!
Walter brought up the same argument at some point. He compared array
literals with string literals. No! String literals only contain
statically-known characters. Array literals may contain anything.
Andrei
But you can simply define:
T[] makeArray(T)(T[] vars...) { return vars; }
auto arr = makeArray(w1, w2, w3);
Unless we make arrays immutable, I don't know how we can define an array
of compile-time constants.
In case this isn't clear:
real [] sinsTable = [ sin(1.0), sin(2.0), sin(3.0), sin(4.0) ];
How do you do this so that the entries in the table are calculated at
compile time?
Re: T[new] misgivings
Don wrote: In case this isn't clear: real [] sinsTable = [ sin(1.0), sin(2.0), sin(3.0), sin(4.0) ]; How do you do this so that the entries in the table are calculated at compile time? static? Andrei
Re: T[new] misgivings
Lutger wrote:
Just to understand it:
int[new] a;
int[new] b;
a = [1,2,3];
b = a;
In your book, the last statement would copy contents of a into b and b.ptr
!= a.ptr while according to walter, b would rebind to a?
Well no.
In the case above b would rebind to a, which is consistent with:
int[new] a = [1, 2, 3];
void fun(int[new] b) { ... }
fun(a); // does not copy a
I am not concerned about assigning one array to another. I'm more
concerned about assigning an array literal to an array.
Andrei
Re: T[new] misgivings
Andrei Alexandrescu wrote: Don wrote: In case this isn't clear: real [] sinsTable = [ sin(1.0), sin(2.0), sin(3.0), sin(4.0) ]; How do you do this so that the entries in the table are calculated at compile time? static? Andrei That's still not compile time. They're initialized in the module constructor. If you make them an enum array, they're at compile time, but then you shouldn't be able to index the array at runtime (the whole point of 'enum' was that it doesn't get stored).
Re: T[new] misgivings
On Fri, 16 Oct 2009 09:00:27 -0500, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
Don wrote:
Max Samukha wrote:
On Thu, 15 Oct 2009 21:55:07 -0500, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
I talked to Walter about T[new] today and it seems we are having a
disagreement.
I'd prefer Walter's way with a provision that array literals are
immutable and allocated statically:
immutable(int)[] a = [1, 2, 3]; // no allocation here
int[new] b = [1, 2]; // new storage is allocated and the literal is
copied there
int[] a = [1, 2, 3]; // error. dup needed
auto c = [1, 2, 3]; // c is of type immutable(int)[]
b[] = c; //b's length is changed and c's contents copied to b's
storage
auto d = [1, 2, 3].dup; // d is of type int[new]
auto e = [1, 2, 3].idup; // e is of type immutable(int)[new]
// arrays are true reference types
int[new] a = [1, 2, 3];
b = a;
a.length = 22;
assert (a.length == b.length);
This makes perfect sense to me. The rule would be:
If 'x' is T[new], then:
x = y; _always_ copies y into a {length, capacity-specified block},
unless it already is one. x is given a pointer to the start of that block.
x[] = y[]; does a memcpy, regardless of whether y is a T[new] or a T[].
It makes sense to make array literals immutable. The trouble is you
won't be able to create arrays of most interesting types that way.
class Widget { ... }
Widget w1, w2, w3;
auto arr = [ w1, w2, w3 ]; // error!
Walter brought up the same argument at some point. He compared array
literals with string literals. No! String literals only contain
statically-known characters. Array literals may contain anything.
Andrei
Ok. But it is unacceptable to allocate the literal on heap if all its
elements are statically known. I'd rather have a library function than
the current situation. Why not:
auto arr = array(w1, w2, w3)?
This is an important issue. Please don't leave it unaddressed.
Re: T[new] misgivings
On Fri, 16 Oct 2009 18:30:24 +0400, Andrei Alexandrescu
seewebsiteforem...@erdani.org wrote:
Lutger wrote:
Just to understand it:
int[new] a;
int[new] b;
a = [1,2,3];
b = a;
In your book, the last statement would copy contents of a into b and
b.ptr != a.ptr while according to walter, b would rebind to a?
Well no.
In the case above b would rebind to a, which is consistent with:
int[new] a = [1, 2, 3];
void fun(int[new] b) { ... }
fun(a); // does not copy a
I am not concerned about assigning one array to another. I'm more
concerned about assigning an array literal to an array.
Andrei
So, the real question is, what's the type of an array literal?
If it's T[new] then you must reassign a reference to be consistent:
int[new] a = arrayLiteral;
int[new] b = a; // reassign reference
foo(arrayLiteral); // a reference passed
Else, invoke opAssign. And its semantics is not clear.
I'm in favor of removing hidden allocations and making compile-time
literals immutable:
immutable(int)[] a1 = [1, 2, 3];
In case of compile-time literal assignment you can't just re-assign a
reference (because it's immutable), so opAssign should take care of it:
int[new] a2;
a2 = [1, 2, 3]; // rewritten as a2.opAssign([1, 2, 3]);
Since there should be now hidden allocation, values should be overwritten.
And there are two options:
1) Strict assignment:
# assert(a2.length == [1, 2, 3].length);
# a2[0..3] = [1, 2, 3];
2) Auto-expand (but not shrink!):
# if (a2.capacity [1, 2, 3].length) throw new Exception(...);
# a2.capacity = [1, 2, 3].length;
# a2[0..3] = [1, 2, 3];
Why not shrink? Because I believe the following test should be valid:
a2 = [1, 2, 3];
assert(a2 == [1, 2, 3]); // they must match if the statement above succeeds
Re: T[new] misgivings
Andrei Alexandrescu, el 16 de octubre a las 09:12 me escribiste: Max Samukha wrote: On Fri, 16 Oct 2009 14:25:44 +0200, Don nos...@nospam.com wrote: Yes, but you could allocate the data immediately after the Array structure, so you only have one allocation. And in the common case, where it never exceeds the original capacity, they stay together and preserve cache locality. void *data; // = raw_data; size_t length; size_t capacity; // = 512 ubyte[512] raw_data; Great! And if the length later exceeds the capacity, try to reallocate in place. If impossible, allocate a new block, copy the data, adjust the data pointer and shrink the original block to the size of the Array struct. Right? That's the idea. The only problem that Walter pointed out was that moving GCs may have a problem with internal pointers. GC have to support internal pointers anyways, I don't see how this changes anything... -- Leandro Lucarella (AKA luca) http://llucax.com.ar/ -- GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145 104C 949E BFB6 5F5A 8D05) -- We're rotten fruit We're damaged goods What the hell, we've got nothing more to lose One gust and we will probably crumble We're backdrifters
Re: T[new] misgivings
Andrei Alexandrescu Wrote: Walter Bright wrote: I think it would be very strange to have T[] behave like a reference type (which it does now) and T[new] to behave like a value type. T[] is not a reference type. Andrei While true, normal use will be the same as if it was a reference type. All legal changes to a slice impact all copies of the slice.
T[new] misgivings
I talked to Walter about T[new] today and it seems we are having a disagreement. The problem is that I believe T[new] is a container, whereas Walter believes T[new] is nothing but a slice with a couple of extra operations. Paradoxically this seems to be conducive to subtle efficiency issues. For example, consider: int[new] a; ... a = [1, 2, 3]; What should that do? Walter: T[new] is a slice with benefits, assignment for slices rebinds the slice, therefore the assignment must do the same. In this case, the assignments allocate a new array and make a refer to that array. Whatever old array a referred to will continue to live wherever it was. Me: T[new] is a container, therefore the assignment must resize the container from whatever size it had to 3 and then write 1, 2, 3 to its three slots. I guess each of us has a point, but this is a setup for an increasingly unpleasant situation. Here's the dialog as it happened. A: Ok, then how do I say the common operation I want to overwrite whatever the array had with 1, 2, 3? I can only presume there must be an obvious and simple way to do so, and I thought a = [1, 2, 3] was the obvious syntax to achieve that. W: No, you must write a[] = [1, 2, 3]; A: But that only works if the container already had length 3. So what I need to do is this: a.length = 3; a[] = [1, 2, 3]; A: But that is inefficient if the array had length less than 3 because it means double assignment W: Nobody complained about it with slices. A: So if I do want something that does the obvious operation Whatever that array had, make it now have 1, 2, 3 as it contents at a reasonable cost I need to call an elaborate function that is highly nontrivial to write? W: Looks like so. A: So then the first Effective D standard would have Item #1: Avoid assignment to arrays. Call the assign() function? W: Nobody complained about it with slices. === This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Andrei
Re: T[new] misgivings
Andrei Alexandrescu wrote: int[new] a; a = [1, 2, 3]; What should that do? This question can be rephrased as, should 'int[new]' be a reference type or a value type (or something else)? If 'int[new]' is a reference type, then it must rebind, because that's what assignment does for reference types. If 'int[new]' is a value type, then it must modify the array in place, because that's all it can do. If 'int[new]' is neither a reference type nor a value type, then we're back to (some of) the problems with slices. To answer the rephrased question: 'int[new]' should be a value type. W: Nobody complained about it with slices. FWIW, I found arrays in D1 so completely broken that I didn't it worth the effort to complain about every little detail. Everything about them was wrong. I consider them a textbook example of what not to do. -- Rainer Deyke - rain...@eldwood.com
Re: T[new] misgivings
Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new]
Re: T[new] misgivings
On Thu, 15 Oct 2009 23:16:56 -0400, Walter Bright newshou...@digitalmars.com wrote: Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new] I agree.
Re: T[new] misgivings
Walter Bright wrote: Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new] Then your argument building on similarity between the two is weakened. T[new] a; T[] b; ... a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Andrei
Re: T[new] misgivings
Rainer Deyke wrote: Andrei Alexandrescu wrote: int[new] a; a = [1, 2, 3]; What should that do? This question can be rephrased as, should 'int[new]' be a reference type or a value type (or something else)? If 'int[new]' is a reference type, then it must rebind, because that's what assignment does for reference types. If 'int[new]' is a value type, then it must modify the array in place, because that's all it can do. If 'int[new]' is neither a reference type nor a value type, then we're back to (some of) the problems with slices. To answer the rephrased question: 'int[new]' should be a value type. Well Walter and I agreed they should be pass-by-reference. That doesn't mean they must be references, and the fact that the simplest syntax has the worst efficiency reminds me of iostreams. W: Nobody complained about it with slices. FWIW, I found arrays in D1 so completely broken that I didn't it worth the effort to complain about every little detail. Everything about them was wrong. I consider them a textbook example of what not to do. My perception is that you're in a minority. Anyway, if there's something that T[new] can help with, let us know. Andrei
Re: T[new] misgivings
On Thu, 15 Oct 2009 22:55:07 -0400, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: I talked to Walter about T[new] today and it seems we are having a disagreement. The problem is that I believe T[new] is a container, whereas Walter believes T[new] is nothing but a slice with a couple of extra operations. Paradoxically this seems to be conducive to subtle efficiency issues. For example, consider: int[new] a; ... a = [1, 2, 3]; What should that do? Walter: T[new] is a slice with benefits, assignment for slices rebinds the slice, therefore the assignment must do the same. In this case, the assignments allocate a new array and make a refer to that array. Whatever old array a referred to will continue to live wherever it was. Me: T[new] is a container, therefore the assignment must resize the container from whatever size it had to 3 and then write 1, 2, 3 to its three slots. I guess each of us has a point, but this is a setup for an increasingly unpleasant situation. Here's the dialog as it happened. A: Ok, then how do I say the common operation I want to overwrite whatever the array had with 1, 2, 3? I can only presume there must be an obvious and simple way to do so, and I thought a = [1, 2, 3] was the obvious syntax to achieve that. W: No, you must write a[] = [1, 2, 3]; A: But that only works if the container already had length 3. So what I need to do is this: a.length = 3; a[] = [1, 2, 3]; A: But that is inefficient if the array had length less than 3 because it means double assignment W: Nobody complained about it with slices. A: So if I do want something that does the obvious operation Whatever that array had, make it now have 1, 2, 3 as it contents at a reasonable cost I need to call an elaborate function that is highly nontrivial to write? W: Looks like so. A: So then the first Effective D standard would have Item #1: Avoid assignment to arrays. Call the assign() function? W: Nobody complained about it with slices. === This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Andrei I like (and have used) the opSliceAssign syntax to represent by value/copy assignment as opposed to opAssign's by reference syntax. You could always define T[new] auto-resize in the case of a[] = b, but then you'd have to decide if that behavior should be extended to slices.
Re: T[new] misgivings
Andrei Alexandrescu Wrote: I talked to Walter about T[new] today and it seems we are having a disagreement. The problem is that I believe T[new] is a container, whereas Walter believes T[new] is nothing but a slice with a couple of extra operations. Paradoxically this seems to be conducive to subtle efficiency issues. For example, consider: int[new] a; ... a = [1, 2, 3]; What should that do? Allocate an array Walter: T[new] is a slice with benefits, assignment for slices rebinds the slice, therefore the assignment must do the same. In this case, the assignments allocate a new array and make a refer to that array. Whatever old array a referred to will continue to live wherever it was. Me: T[new] is a container, therefore the assignment must resize the container from whatever size it had to 3 and then write 1, 2, 3 to its three slots. I guess each of us has a point, but this is a setup for an increasingly unpleasant situation. Here's the dialog as it happened. A: Ok, then how do I say the common operation I want to overwrite whatever the array had with 1, 2, 3? I can only presume there must be an obvious and simple way to do so, and I thought a = [1, 2, 3] was the obvious syntax to achieve that. W: No, you must write a[] = [1, 2, 3]; That matches my expectation A: But that only works if the container already had length 3. So what I need to do is this: a.length = 3; a[] = [1, 2, 3]; A: But that is inefficient if the array had length less than 3 because it means double assignment The optimizer should be able to make that efficient.
Re: T[new] misgivings
Andrei Alexandrescu wrote: I talked to Walter about T[new] today and it seems we are having a disagreement. The problem is that I believe T[new] is a container, whereas Walter believes T[new] is nothing but a slice with a couple of extra operations. I agree with the container model, it should work the way std.array.Appender does right now. T[new] would be used when you need to grow the array, and t[] when the size is final. Paradoxically this seems to be conducive to subtle efficiency issues. For example, consider: int[new] a; ... a = [1, 2, 3]; What should that do? Walter: T[new] is a slice with benefits, assignment for slices rebinds the slice, therefore the assignment must do the same. In this case, the assignments allocate a new array and make a refer to that array. Whatever old array a referred to will continue to live wherever it was. Me: T[new] is a container, therefore the assignment must resize the container from whatever size it had to 3 and then write 1, 2, 3 to its three slots. I think Walter wants to keep the syntax consistent with slices, even if T[new] is a container, it would just mean assign this new slice to the container. I guess each of us has a point, but this is a setup for an increasingly unpleasant situation. Here's the dialog as it happened. A: Ok, then how do I say the common operation I want to overwrite whatever the array had with 1, 2, 3? I can only presume there must be an obvious and simple way to do so, and I thought a = [1, 2, 3] was the obvious syntax to achieve that. W: No, you must write a[] = [1, 2, 3]; A: But that only works if the container already had length 3. So what I need to do is this: a.length = 3; a[] = [1, 2, 3]; A: But that is inefficient if the array had length less than 3 because it means double assignment If the array is of type T[new] then the runtime implementation (_d_array_new_assign?) would take care of resizing the array. As opposed to slices which can't add/remove memory from arrays anymore. You therefore wouldn't need to set the length before assignments. W: Nobody complained about it with slices. A: So if I do want something that does the obvious operation Whatever that array had, make it now have 1, 2, 3 as it contents at a reasonable cost I need to call an elaborate function that is highly nontrivial to write? W: Looks like so. Why would you need a nontrivial function? It's all hidden in the compiler runtime implementation, which would just get uninitialized memory if needed and then copy the assignment, its fairly simple. A: So then the first Effective D standard would have Item #1: Avoid assignment to arrays. Call the assign() function? W: Nobody complained about it with slices. === This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Raise a compiler error, its unsafe because its unknown of b is an entire array (so .ptr is a valid GC root) or a slice. T[new] would implicitly cast to T[] but the opposite requires a cast. T[new] a; T[] b; a = cast(T[new])b; // safe because we assume b.ptr is a valid gc root a = b.dup; // safe because the compiler can infer its a gc root Jeremie
Re: T[new] misgivings
Robert Jacques wrote: I like (and have used) the opSliceAssign syntax to represent by value/copy assignment as opposed to opAssign's by reference syntax. You could always define T[new] auto-resize in the case of a[] = b, but then you'd have to decide if that behavior should be extended to slices. I could. Walter doesn't wanna. He says he wants a[] = b[] to generate an unchecked memcpy in release mode. Andrei
Re: T[new] misgivings
Andrei Alexandrescu wrote: Rainer Deyke wrote: To answer the rephrased question: 'int[new]' should be a value type. Well Walter and I agreed they should be pass-by-reference. That doesn't mean they must be references, and the fact that the simplest syntax has the worst efficiency reminds me of iostreams. Given that D already has both value types and reference types, the addition of types that are passed by reference but otherwise act as value types actually seems reasonable. It make the language more orthogonal. Classes have one set of attributes. Structs have another. If the language absolutely needs to support both sets of attributes, I should at least be able to mix and match between them. So, what's the syntax for user-defined value types that are passed by reference going to be? ref struct? opPass? FWIW, I found arrays in D1 so completely broken that I didn't it worth the effort to complain about every little detail. Everything about them was wrong. I consider them a textbook example of what not to do. My perception is that you're in a minority. Anyway, if there's something that T[new] can help with, let us know. I was under the impression that arrays were generally considered broken, which is why 'T[new]' is now being introduced. -- Rainer Deyke - rain...@eldwood.com
Re: T[new] misgivings
Andrei Alexandrescu wrote: Then your argument building on similarity between the two is weakened. T[new] a; T[] b; a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Actually [1, 2, 3] looks more like an array than a slice to me. Arrays can be assigned to slices, no? -- Rainer Deyke - rain...@eldwood.com
Re: T[new] misgivings
== Quote from Andrei Alexandrescu (seewebsiteforem...@erdani.org)'s article
I talked to Walter about T[new] today and it seems we are having a
disagreement.
The problem is that I believe T[new] is a container, whereas Walter
believes T[new] is nothing but a slice with a couple of extra operations.
Paradoxically this seems to be conducive to subtle efficiency issues.
For example, consider:
int[new] a;
...
a = [1, 2, 3];
What should that do?
Walter: T[new] is a slice with benefits, assignment for slices rebinds
the slice, therefore the assignment must do the same. In this case, the
assignments allocate a new array and make a refer to that array.
Whatever old array a referred to will continue to live wherever it was.
But isn't part of the point of T[new] that you're supposed to only have one
T[new]
pointing to any given block of memory? If you can bind a slice to a T[new],
then
you can bind a slice to multiple T[new]s. Then you get back to having weird
bugs
like:
immutable(int)[new] foo;
foreach(i; 0..5) {
foo ~= i;
}
immutable(int)[] bar = foo[0..4];
immutable(int)[new] baz = bar; // References the same memory as foo
baz ~= 666;
writeln(foo); // prints [1 2 3 4 666].
The only way around this would be something like COW semantics, i.e. you can
assign a slice to a T[new] w/o copying, but when you try to do anything with it
that couldn't be done w/ a slice, then it copies.
Re: T[new] misgivings
Rainer Deyke wrote: Andrei Alexandrescu wrote: Rainer Deyke wrote: To answer the rephrased question: 'int[new]' should be a value type. Well Walter and I agreed they should be pass-by-reference. That doesn't mean they must be references, and the fact that the simplest syntax has the worst efficiency reminds me of iostreams. Given that D already has both value types and reference types, the addition of types that are passed by reference but otherwise act as value types actually seems reasonable. It make the language more orthogonal. Classes have one set of attributes. Structs have another. If the language absolutely needs to support both sets of attributes, I should at least be able to mix and match between them. So, what's the syntax for user-defined value types that are passed by reference going to be? ref struct? opPass? No need for new syntax. T[new] is a struct that has a pointer inside. FWIW, I found arrays in D1 so completely broken that I didn't it worth the effort to complain about every little detail. Everything about them was wrong. I consider them a textbook example of what not to do. My perception is that you're in a minority. Anyway, if there's something that T[new] can help with, let us know. I was under the impression that arrays were generally considered broken, which is why 'T[new]' is now being introduced. There is a pernicious issue with ~= and slices. T[new] aims at fixing that. Andrei
Re: T[new] misgivings
Rainer Deyke wrote: Andrei Alexandrescu wrote: Then your argument building on similarity between the two is weakened. T[new] a; T[] b; a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Actually [1, 2, 3] looks more like an array than a slice to me. Arrays can be assigned to slices, no? My point exactly. Andrei
Re: T[new] misgivings
Andrei Alexandrescu wrote: Walter Bright wrote: Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new] Then your argument building on similarity between the two is weakened. T[new] a; T[] b; ... a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Andrei Simple, assignment to a fails 'cannot cast T[3] to T[new]'. It's already consistent with slices of different types: char[] a = foo; // error, cannot cast immutable(char)[] to char[] int[new] b = [1, 2, 3]; // error, cannot cast int[3] to int[new] you have to do: char[] a = foo.dup; int[new] b = [1, 2, 3].dup; Jeremie
Re: T[new] misgivings
Jeremie Pelletier wrote: Andrei Alexandrescu wrote: Walter Bright wrote: Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new] Then your argument building on similarity between the two is weakened. T[new] a; T[] b; ... a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Andrei Simple, assignment to a fails 'cannot cast T[3] to T[new]'. It's already consistent with slices of different types: char[] a = foo; // error, cannot cast immutable(char)[] to char[] int[new] b = [1, 2, 3]; // error, cannot cast int[3] to int[new] you have to do: char[] a = foo.dup; int[new] b = [1, 2, 3].dup; Jeremie I'd be _very_ unhappy to have to explain to people how in the world we managed to make the most intuitive syntax not work at all. Andrei
Re: T[new] misgivings
Andrei Alexandrescu wrote: Jeremie Pelletier wrote: Andrei Alexandrescu wrote: Walter Bright wrote: Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new] Then your argument building on similarity between the two is weakened. T[new] a; T[] b; ... a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Andrei Simple, assignment to a fails 'cannot cast T[3] to T[new]'. It's already consistent with slices of different types: char[] a = foo; // error, cannot cast immutable(char)[] to char[] int[new] b = [1, 2, 3]; // error, cannot cast int[3] to int[new] you have to do: char[] a = foo.dup; int[new] b = [1, 2, 3].dup; Jeremie I'd be _very_ unhappy to have to explain to people how in the world we managed to make the most intuitive syntax not work at all. Andrei I agree it can be confusing, the first time i tried to assign a string literal to a char[] in D2 I had to pause for a second to understand what was happening :) But what I don't like is that assigning memory from the static data segment to a resizable array isn't safe. Unless the GC can detect that the memory it is trying to resize isn't part of the heap and automatically create a new allocation for it, you're gonna have nasty side effects. The compiler could also implicitly copy the slice, but then it should also automatically copy a foo literal when assigned to char[] to keep consistent. Jeremie
Re: T[new] misgivings
Jeremie Pelletier wrote: Andrei Alexandrescu wrote: Jeremie Pelletier wrote: Andrei Alexandrescu wrote: Walter Bright wrote: Andrei Alexandrescu wrote: This goes into something more interesting that I thought of after the conversation. Consider: T[new] a; T[] b; ... a = b; What should that do? Error. T[] cannot be implicitly converted to T[new] Then your argument building on similarity between the two is weakened. T[new] a; T[] b; ... a = [1, 2, 3]; b = [1, 2, 3]; Central to your argument was that the two must do the same thing. Since now literals are in a whole new league (they aren't slices because slices can't be assigned to arrays), the cornerstone of your argument goes away. Andrei Simple, assignment to a fails 'cannot cast T[3] to T[new]'. It's already consistent with slices of different types: char[] a = foo; // error, cannot cast immutable(char)[] to char[] int[new] b = [1, 2, 3]; // error, cannot cast int[3] to int[new] you have to do: char[] a = foo.dup; int[new] b = [1, 2, 3].dup; Jeremie I'd be _very_ unhappy to have to explain to people how in the world we managed to make the most intuitive syntax not work at all. Andrei I agree it can be confusing, the first time i tried to assign a string literal to a char[] in D2 I had to pause for a second to understand what was happening :) But what I don't like is that assigning memory from the static data segment to a resizable array isn't safe. Unless the GC can detect that the memory it is trying to resize isn't part of the heap and automatically create a new allocation for it, you're gonna have nasty side effects. The compiler could also implicitly copy the slice, but then it should also automatically copy a foo literal when assigned to char[] to keep consistent. Jeremie Speaking of which, a funny fallout of this a = [1, 2, 3] thing is that we, while striving to avoid all hidden allocations, ended up doing the worst hidden allocation with the simplest and most intuitive syntax. Andrei
