Re: Casting away immutability
On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev wrote: [...] Agree, however, memory obtained with mmap(..., PROT_READ, ..); is essentially read-only and any attempt to write to it will cause immediate crash with segmentation violation. It would be very desirable in this case to return something that could be cast to immutable(type)[] but not type[]. This is what I tried to find out. Thank you for the help. It can still change by writing (in the same or another process) to the file via another file descriptor. Of course the results are non-deterministic in that case, so it doesn't much matter whether the compiler assumes the data to be immutable or not.
Re: Casting away immutability
On Friday, 4 September 2015 at 08:37:26 UTC, Kagamin wrote: On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev wrote: Agree, however, memory obtained with mmap(..., PROT_READ, ..); is essentially read-only and any attempt to write to it will cause immediate crash with segmentation violation. It would be very desirable in this case to return something that could be cast to immutable(type)[] but not type[]. If you can't write to it doesn't mean it won't change. Use const for memory that you can't write to. Note that you can however achieve immutability by using a _private_ read-only mapping.
Re: Casting away immutability
On Friday, 4 September 2015 at 09:05:07 UTC, Marc Schütz wrote: Note that you can however achieve immutability by using a _private_ read-only mapping. man pages say the behavior is unspecified.
Re: Casting away immutability
On Friday, 4 September 2015 at 13:29:42 UTC, Kagamin wrote: On Friday, 4 September 2015 at 09:05:07 UTC, Marc Schütz wrote: Note that you can however achieve immutability by using a _private_ read-only mapping. man pages say the behavior is unspecified. The Linux man page says: "It is unspecified whether changes made to the file after the mmap() call are visible in the mapped region." I.e., you're right. I interpreted this to refer only to portions of the file that have never been accessed, but reading it again, it doesn't say that... And it makes sense, as this makes it possible to drop clean pages on memory pressure, instead of swapping them out.
Re: Casting away immutability
On Wednesday, 2 September 2015 at 04:04:54 UTC, Sergei Degtiarev wrote: I seems a little bit too easy to to shoot yourself in the foot. Yes, cast is a little not fool-proof in this scenario. On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev wrote: Agree, however, memory obtained with mmap(..., PROT_READ, ..); is essentially read-only and any attempt to write to it will cause immediate crash with segmentation violation. It would be very desirable in this case to return something that could be cast to immutable(type)[] but not type[]. If you can't write to it doesn't mean it won't change. Use const for memory that you can't write to. On Thursday, 3 September 2015 at 16:05:33 UTC, Sergei Degtiarev wrote: Absolutely, instead of returning raw void[] and allow user to cast it, std.mmfile should implement template function to return desired type. Well, interpretation of data can be arbitrarily complex, I'd suggest that the raw data should be passed to your specific parser, that will do the right thing and provide data in expected format.
Re: Casting away immutability
On Thursday, 3 September 2015 at 13:28:54 UTC, Sergei Degtiarev
wrote:
Agree, however, memory obtained with mmap(..., PROT_READ, ..);
is essentially read-only and any attempt to write to it will
cause immediate crash with segmentation violation. It would be
very desirable in this case to return something that could be
cast to immutable(type)[] but not type[].
This is what I tried to find out. Thank you for the help.
immutable(T)[] getGetData(T)() {
return cast(immutable(T)[])data;
}
...
auto ints = obj.getData!int;
Re: Casting away immutability
On Thursday, 3 September 2015 at 14:36:12 UTC, Mike Parker wrote:
immutable(T)[] getGetData(T)() {
return cast(immutable(T)[])data;
}
Absolutely, instead of returning raw void[] and allow user to
cast it, std.mmfile should implement template function to return
desired type.
This would allow all necessary checks inside the module,
returning pure result.
Re: Casting away immutability
On Thursday, 3 September 2015 at 05:15:35 UTC, Jonathan M Davis wrote: On Wednesday, September 02, 2015 14:00:07 Sergei Degtiarev via Digitalmars-d-learn wrote: Well, that's how mmap works. You're just getting raw bytes as void*, and the D code converts that to void[], which is slightly safer. It doesn't inherently have a type, and often, the data referred to by mmap never did have a type. It's just bytes in a file. So, if you want to use it as a type, you have to tell the compiler what it's type is - and get it right - via a cast. And yes, you can cast to immutable as part of that, because casts let you shoot yourself in the foot, because you're telling the compiler that you know better than it does, but you shouldn't be casting anything you get from C to immutable, because it's not going to be immutable. Most code shouldn't be casting to immutable any more than it should be casting away immutable. immutable data is treated by the compiler as immutable - it will _never_ change over the course of the program - so if it does change, you're going to have fun bugs, and if it really refers to mutable data (as would be the case with an mmapped file), then it could change. Additionally, immutable is treated as implicitly shared, so the compiler could do different optimizations based on that (though that probably won't affect anything if you only ever have it one a single thread), and if it's not really immutable, you could have fun bugs caused by that. Don't cast to or from immutable unless you know what you're doing, and in most of those cases, there's a decent chance that that's not the best way to do what you're trying to do anyway. Agree, however, memory obtained with mmap(..., PROT_READ, ..); is essentially read-only and any attempt to write to it will cause immediate crash with segmentation violation. It would be very desirable in this case to return something that could be cast to immutable(type)[] but not type[]. This is what I tried to find out. Thank you for the help.
Re: Casting away immutability
On Wednesday, September 02, 2015 14:00:07 Sergei Degtiarev via Digitalmars-d-learn wrote: > On Wednesday, 2 September 2015 at 04:19:24 UTC, lobo wrote: > > No, I think your design is unsafe because you're throwing away > > type information and returning void[], then telling the > > compiler not to worry about it. > It is unsafe, but this not my design but std.mmfile module in > phobos. This is what I'm trying to improve. And there was no type > information initially, this is raw allocated memory returned to > user. Well, that's how mmap works. You're just getting raw bytes as void*, and the D code converts that to void[], which is slightly safer. It doesn't inherently have a type, and often, the data referred to by mmap never did have a type. It's just bytes in a file. So, if you want to use it as a type, you have to tell the compiler what it's type is - and get it right - via a cast. And yes, you can cast to immutable as part of that, because casts let you shoot yourself in the foot, because you're telling the compiler that you know better than it does, but you shouldn't be casting anything you get from C to immutable, because it's not going to be immutable. Most code shouldn't be casting to immutable any more than it should be casting away immutable. immutable data is treated by the compiler as immutable - it will _never_ change over the course of the program - so if it does change, you're going to have fun bugs, and if it really refers to mutable data (as would be the case with an mmapped file), then it could change. Additionally, immutable is treated as implicitly shared, so the compiler could do different optimizations based on that (though that probably won't affect anything if you only ever have it one a single thread), and if it's not really immutable, you could have fun bugs caused by that. Don't cast to or from immutable unless you know what you're doing, and in most of those cases, there's a decent chance that that's not the best way to do what you're trying to do anyway. But at the end of the day, the cast operator is a blunt instrument which inloves you telling the compiler that you know better than it does, so you had better know better, or you're going to shoot yourself in the foot. - Jonathan M Davis
Re: Casting away immutability
On Wednesday, 2 September 2015 at 04:19:24 UTC, lobo wrote: No, I think your design is unsafe because you're throwing away type information and returning void[], then telling the compiler not to worry about it. It is unsafe, but this not my design but std.mmfile module in phobos. This is what I'm trying to improve. And there was no type information initially, this is raw allocated memory returned to user.
Re: Casting away immutability
On Wednesday, 2 September 2015 at 02:05:00 UTC, Sergei Degtiarev wrote: I can't understand how cast coexist with immutability. Cast bypasses immutability, triggering implementation-defined behavior. You might modify immutable data, you might cause the program to crash, it might just not do anything, depending on exactly what is going on in the implementation. When you use cast, you basically take matters into your own hands.
Re: Casting away immutability
On Wednesday, September 02, 2015 02:04:58 Sergei Degtiarev via Digitalmars-d-learn wrote: > I can't understand how cast coexist with immutability. Consider > the code: > immutable immutable(int)[4] buf; > auto x=buf[0]; > auto p=buf.ptr; > > auto i=cast(int[]) buf; > i[]=1; > > assert(buf.ptr == p); > assert(buf[0] != x); > > I've just modified immutable data, is it normal? > Another, ever simpler, code: > immutable a=0; > auto b=cast(int*) > *b=1; > assert( == b); > assert(a != *b); > Last two assertions both succeed and really puzzle me. What is > pointer in D at all? You can cast away immutable, just like you can cast away const, but in either case, mutating the data after casting it to mutable is undefined behavior. The compiler is free to assume that you won't do that, and in the case of immutable, if the data is actually in ROM, it could crash your program. The two main places that casting away const or immutable would be used at this point would be to pass it to a C function which isn't properly annotated (but which you know won't mutate the data) or when you cast to immutable to pass something via std.concurrency and then to cast back to mutable on the receiving thread, since we don't currently have a way to indicate ownership in a manner that would allow us to pass mutable data across threads. But if you do that, you need to be sure that you just passed across the only reference to that data, or you're going to have bugs, because anything which isn't immutable or shared is considered thread-local by the compiler. So, there are cases where it makes sense if you're very careful and know what you're doing, but those cases are few and far between. But D allows the cast if nothing else, because it's a systems language and will let you shoot yourself in the foot if you really try to. And when you cast, you're telling the compiler that you know better than it does. So, you'd better be right, or you're going to blow your foot off (figuratively speaking). Regardless, casting away const or immutable and then mutating is undefined behavior. So, don't do it. - Jonathan M Davis
Casting away immutability
I can't understand how cast coexist with immutability. Consider the code: immutable immutable(int)[4] buf; auto x=buf[0]; auto p=buf.ptr; auto i=cast(int[]) buf; i[]=1; assert(buf.ptr == p); assert(buf[0] != x); I've just modified immutable data, is it normal? Another, ever simpler, code: immutable a=0; auto b=cast(int*) *b=1; assert( == b); assert(a != *b); Last two assertions both succeed and really puzzle me. What is pointer in D at all?
Re: Casting away immutability
On Wednesday, 2 September 2015 at 04:04:54 UTC, Sergei Degtiarev wrote: On Wednesday, 2 September 2015 at 02:50:30 UTC, Jonathan M Davis wrote: is undefined behavior. So, don't do it. I don't. Actually, I'm looking for opposite - to protect data, like it is a class with two methods, returning void[] and immutable(void)[] as memory buffer, which user of the class supposed to cast to desired type. I naively assumed that void[] to be cast to, say, int[]; and immutable(void)[] to immutable(int)[] respectively. Unfortunately, D cast notices no difference between these two types. I seems a little bit too easy to to shoot yourself in the foot. No, I think your design is unsafe because you're throwing away type information and returning void[], then telling the compiler not to worry about it. And still, assert( == b); assert(a != *b); How these two lines would look in assembler? Defined behavior or undefined behavior, I can't imagine that. why don't you have a look? bye, lobo
Re: Casting away immutability
On Wednesday, 2 September 2015 at 02:50:30 UTC, Jonathan M Davis wrote: is undefined behavior. So, don't do it. I don't. Actually, I'm looking for opposite - to protect data, like it is a class with two methods, returning void[] and immutable(void)[] as memory buffer, which user of the class supposed to cast to desired type. I naively assumed that void[] to be cast to, say, int[]; and immutable(void)[] to immutable(int)[] respectively. Unfortunately, D cast notices no difference between these two types. I seems a little bit too easy to to shoot yourself in the foot. And still, assert( == b); assert(a != *b); How these two lines would look in assembler? Defined behavior or undefined behavior, I can't imagine that.
