Re: @safe std.file.read
On Monday, January 6, 2020 8:52:01 AM MST Steven Schveighoffer via Digitalmars-d-learn wrote: > On 1/6/20 5:07 AM, WebFreak001 wrote: > > I was wondering, how are you supposed to use std.file : read in @safe > > code when it returns a void[] but you want to get all bytes in the file? > > > > Is void[] really the correct type it should be returning instead of > > ubyte[] when it just reads a (binary) file to memory? Or should void[] > > actually be castable to ubyte[] in @safe code? > > I feel like this conversation has been had before. But I think it should > be ubyte[]. Not sure why it's void[]. Perhaps for symmetry with write, > which takes void[] (for good reason)? I think that in previous discussions, it was decided that in general, when you're dealing with something like reading from / write to a file or a socket, writing should accept void[], because then you can write any binary data to it without casting (including objects which are being serialized), whereas reading should give you ubyte[] or const(ubyte)[], because what you're getting from the OS is bytes of data, and it's up to the program to figure out what to do with them. - Jonathan M Davis
Re: @safe std.file.read
On 1/6/20 5:07 AM, WebFreak001 wrote: Or should void[] actually be castable to ubyte[] in @safe code? No, because you can implicitly cast anything to void[], including pointer arrays. Possibly const(ubyte[]). -Steve
Re: @safe std.file.read
On 1/6/20 5:07 AM, WebFreak001 wrote: I was wondering, how are you supposed to use std.file : read in @safe code when it returns a void[] but you want to get all bytes in the file? Is void[] really the correct type it should be returning instead of ubyte[] when it just reads a (binary) file to memory? Or should void[] actually be castable to ubyte[] in @safe code? I feel like this conversation has been had before. But I think it should be ubyte[]. Not sure why it's void[]. Perhaps for symmetry with write, which takes void[] (for good reason)? -Steve
Re: @safe std.file.read
On Monday, 6 January 2020 at 10:07:37 UTC, WebFreak001 wrote: I was wondering, how are you supposed to use std.file : read in @safe code when it returns a void[] but you want to get all bytes in the file? Is void[] really the correct type it should be returning instead of ubyte[] when it just reads a (binary) file to memory? Or should void[] actually be castable to ubyte[] in @safe code? I definitely think it should return ubyte[]. void[] is a very special abstraction that shouldn't be used at all if you don't know very well what you're doing.
Re: @safe std.file.read
I would say it should return a ubyte[]. On Monday, 6 January 2020 at 10:07:37 UTC, WebFreak001 wrote: Or should void[] actually be castable to ubyte[] in @safe code? Definitely not with the current semantics, since a void[] can alias pointers in @safe code. See: https://issues.dlang.org/show_bug.cgi?id=20345
@safe std.file.read
I was wondering, how are you supposed to use std.file : read in @safe code when it returns a void[] but you want to get all bytes in the file? Is void[] really the correct type it should be returning instead of ubyte[] when it just reads a (binary) file to memory? Or should void[] actually be castable to ubyte[] in @safe code?
Re: std.file.read returns void[] why?
On 4/18/14, monarch_dodra via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: Yeah... static assert(void.sizeof == 1); passes :/ Note that you can even have static void arrays. E.g.: https://issues.dlang.org/show_bug.cgi?id=9691 I'm not sure whether this is an oversight (accepts-invalid) or something else. But it needs to be properly documented.
Re: std.file.read returns void[] why?
On Thursday, 17 April 2014 at 21:27:44 UTC, Steven Schveighoffer wrote: On Thu, 17 Apr 2014 17:04:25 -0400, monarch_dodra monarchdo...@gmail.com wrote: void[] will only make sense once you've accepted that void.sizeof == 1. It is already accepted that when we talk about length in a void[], it's the number of bytes. But the data has no formal type. Well, I always thought that void[] slice meant there are slice.length items, starting at slice.ptr. I don't know the size of the individual items. For example, in C, a lot of functions take void* first, size_t num, size_t width. In fact, most of druntime functions take void[] buffers that work that way. There's an associated typeid, so that you can now how large each individual items are. But any array implicitly casts to void[]. This is why it makes a good parameter for read or write (when reading or writing the binary data). I guess. I just find it kind of strange that a type that has no type would have an actual sizeof. Then again, I thought void had no sizeof in C, but I just checked, and I was wrong. Well, I guess void[] is C++'s char* for indiscriminate buffers. Speaking of which, does void* trigger strict aliasing in D? This subject seems like a hot potato no-one wants to touch. No, it's equivalent to void *, not char *. in D, ubyte[] would be the equivalent of C's char *. -Steve Correct.
Re: std.file.read returns void[] why?
On Fri, 18 Apr 2014 02:04:16 -0400, monarch_dodra monarchdo...@gmail.com
wrote:
On Thursday, 17 April 2014 at 21:27:44 UTC, Steven Schveighoffer wrote:
On Thu, 17 Apr 2014 17:04:25 -0400, monarch_dodra
monarchdo...@gmail.com wrote:
void[] will only make sense once you've accepted that void.sizeof ==
1.
It is already accepted that when we talk about length in a void[], it's
the number of bytes. But the data has no formal type.
Well, I always thought that void[] slice meant there are slice.length
items, starting at slice.ptr. I don't know the size of the individual
items.
For example, in C, a lot of functions take void* first, size_t num,
size_t width.
In fact, most of druntime functions take void[] buffers that work that
way. There's an associated typeid, so that you can now how large each
individual items are.
import std.stdio;
void main()
{
int[] x = new int[5];
void[] y = x;
writeln(y.length); // 20
}
When a function takes a typeid, that usually is because the translation is
not made. In druntine cases, the compiler is removing the type, and
sticking it into the typeid instead. But the length has not been
translated to bytes! It's still in terms of the original type.
In those cases, it's equivalent to:
void[] y = *cast(void[]*)x;
which would make y.length == 5.
But any array implicitly casts to void[]. This is why it makes a good
parameter for read or write (when reading or writing the binary data).
I guess. I just find it kind of strange that a type that has no type
would have an actual sizeof. Then again, I thought void had no sizeof in
C, but I just checked, and I was wrong.
It's a little strange, but while void has no size, void[] *does* have a
size. The size is in bytes. You can think of an array as starts at this
address, and ends at that address. Because addresses are in terms of
bytes, so is the length of that array.
I admit, I didn't think C's void had a size ;) I'm pretty sure it doesn't
in D, but then again...
-Steve
Re: std.file.read returns void[] why?
On Friday, 18 April 2014 at 13:08:04 UTC, Steven Schveighoffer wrote: I admit, I didn't think C's void had a size ;) I'm pretty sure it doesn't in D, but then again... -Steve Yeah... static assert(void.sizeof == 1); passes :/ So in any case, long story short: void[]: This is an un-typed buffer, pointing to a memory location that starts at .ptr, and is .length bytes in length. Also, as far as the GC is concerned, void is a type that should be scanned (whether or not the data originally allocated was marked as such is another issue).
Re: std.file.read returns void[] why?
On Wed, 16 Apr 2014 14:36:20 +0100, Spacen Jasset
spacenjas...@mailrazer.com wrote:
Why does the read function return void[] and not byte[]
void[] read(in char[] name, size_t upTo = size_t.max);
One one hand the data is always /actually/ going to be a load of (u)bytes,
but /conceptually/ it might be structs or something else and using void[]
therefore doesn't /imply/ anything about what the data really is.
I also thought that void[] was implicitly cast.. but it seems this either
has never been the case or was changed at some point:
import std.stdio;
void main(string[] args)
{
byte[] barr = new byte[10];
foreach(i, ref b; barr)
b = cast(byte)('a' + i);
void[] varr = barr;
char[] carr;
//carr = barr; // Error: cannot implicitly convert expression (barr) of
type byte[] to char[]
carr = cast(char[])barr;
//carr = varr; // Error: cannot implicitly convert expression (varr) of
type void[] to char[]
carr = cast(char[])varr;
writefln(%d,%s, carr.length, carr);
}
I am curious, was it ever possible, was it changed? why? It's always
safe - as the compiler knows how much data the void[] contains, and
void[] is untyped so it sorta makes sense to allow it..
R
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: std.file.read returns void[] why?
You can implicitly cast /to/ void[], but I don't think you could ever implicitly cast from it. Casting from it needs a bit more thought because you should ensure that it actually is what you are saying it is and the compiler can't help with that, so by prohibiting the automatic cast it at least draws your attention to it.
Re: std.file.read returns void[] why?
On Thu, 17 Apr 2014 07:57:35 -0400, Regan Heath re...@netmail.co.nz
wrote:
On Wed, 16 Apr 2014 14:36:20 +0100, Spacen Jasset
spacenjas...@mailrazer.com wrote:
Why does the read function return void[] and not byte[]
void[] read(in char[] name, size_t upTo = size_t.max);
One one hand the data is always /actually/ going to be a load of
(u)bytes, but /conceptually/ it might be structs or something else and
using void[] therefore doesn't /imply/ anything about what the data
really is.
I also thought that void[] was implicitly cast.. but it seems this
either has never been the case or was changed at some point:
import std.stdio;
void main(string[] args)
{
byte[] barr = new byte[10];
foreach(i, ref b; barr)
b = cast(byte)('a' + i);
void[] varr = barr;
char[] carr;
//carr = barr; // Error: cannot implicitly convert expression (barr) of
type byte[] to char[]
carr = cast(char[])barr;
//carr = varr; // Error: cannot implicitly convert expression (varr) of
type void[] to char[]
carr = cast(char[])varr;
writefln(%d,%s, carr.length, carr);
}
I am curious, was it ever possible, was it changed? why? It's always
safe - as the compiler knows how much data the void[] contains, and
void[] is untyped so it sorta makes sense to allow it..
It was never possible. You must explicitly cast to void[].
void[] makes actually little sense as the result of whole-file read that
allocates. byte[] is at least usable and more accurate. In fact, it's a
little dangerous to use void[], since you could assign pointer-containing
values to the void[] and it should be marked as NOSCAN (no pointers inside
file data).
However, when using the more conventional read(void[]) makes a LOT of
sense, since any T[] implicitly casts to void[].
-Steve
Re: std.file.read returns void[] why?
On Thu, 17 Apr 2014 13:59:20 +0100, Steven Schveighoffer schvei...@yahoo.com wrote: It was never possible. You must explicitly cast to void[]. to - from? void[] makes actually little sense as the result of whole-file read that allocates. byte[] is at least usable and more accurate. In fact, it's a little dangerous to use void[], since you could assign pointer-containing values to the void[] and it should be marked as NOSCAN (no pointers inside file data). I see what you're saying, byte[] is what *is* allocated.. but my point is that it's not what those bytes actually represent. Are you saying void[] *is* currently marked NOSCAN? However, when using the more conventional read(void[]) makes a LOT of sense, since any T[] implicitly casts to void[]. Indeed. :) R -- Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: std.file.read returns void[] why?
On Thu, 17 Apr 2014 10:05:49 -0400, Regan Heath re...@netmail.co.nz wrote: On Thu, 17 Apr 2014 13:59:20 +0100, Steven Schveighoffer schvei...@yahoo.com wrote: It was never possible. You must explicitly cast to void[]. to - from? Yes, sorry :) void[] makes actually little sense as the result of whole-file read that allocates. byte[] is at least usable and more accurate. In fact, it's a little dangerous to use void[], since you could assign pointer-containing values to the void[] and it should be marked as NOSCAN (no pointers inside file data). I see what you're saying, byte[] is what *is* allocated.. but my point is that it's not what those bytes actually represent. Are you saying void[] *is* currently marked NOSCAN? No, I mean the return value from read, since it's newly allocated general data, should be marked NOSCAN. Casting the type does not change how the block is marked, only the allocation type makes that distinction. When you *allocate* a void[] buffer, it's marked no scan. But when you allocate a byte[] buffer and implicitly cast it to void[], it's not marked NOSCAN. TL;DR, IMO read should return byte[]. -Steve
Re: std.file.read returns void[] why?
On Thursday, 17 April 2014 at 14:05:50 UTC, Regan Heath wrote: On Thu, 17 Apr 2014 13:59:20 +0100, Steven Schveighoffer schvei...@yahoo.com wrote: It was never possible. You must explicitly cast to void[]. to - from? void[] makes actually little sense as the result of whole-file read that allocates. byte[] is at least usable and more accurate. In fact, it's a little dangerous to use void[], since you could assign pointer-containing values to the void[] and it should be marked as NOSCAN (no pointers inside file data). I see what you're saying, byte[] is what *is* allocated.. but my point is that it's not what those bytes actually represent. Are you saying void[] *is* currently marked NOSCAN? However, when using the more conventional read(void[]) makes a LOT of sense, since any T[] implicitly casts to void[]. Indeed. :) R auto a1 = new ubyte[10]; //NO_SCAN set auto a2 = new ubyte*[10]; // NO_SCAN not set auto a3 = new void[10]; //NO_SCAN not set auto a4 = new void *[10]; //NO_SCAN not set void [] retains = a1;//NO_SCAN REMAINS SET from the ubyte [] at creation time. Since read comes straight from the file. It contains no memory pointers and the NO_SCAN can be set.
Re: std.file.read returns void[] why?
Are you saying void[] *is* currently marked NOSCAN? import std.stdio; import core.memory; writeln(int* [], GC.query(cast(void *) arr2).attr);
Re: std.file.read returns void[] why?
On Thursday, 17 April 2014 at 12:59:20 UTC, Steven Schveighoffer wrote: It was never possible. You must explicitly cast to void[]. void[] makes actually little sense as the result of whole-file read that allocates. byte[] is at least usable and more accurate. In fact, it's a little dangerous to use void[], since you could assign pointer-containing values to the void[] and it should be marked as NOSCAN (no pointers inside file data). However, when using the more conventional read(void[]) makes a LOT of sense, since any T[] implicitly casts to void[]. -Steve void[] will only make sense once you've accepted that void.sizeof == 1. Well, I guess void[] is C++'s char* for indiscriminate buffers. Speaking of which, does void* trigger strict aliasing in D? This subject seems like a hot potato no-one wants to touch.
Re: std.file.read returns void[] why?
On Thu, 17 Apr 2014 17:04:25 -0400, monarch_dodra monarchdo...@gmail.com wrote: On Thursday, 17 April 2014 at 12:59:20 UTC, Steven Schveighoffer wrote: It was never possible. You must explicitly cast to void[]. void[] makes actually little sense as the result of whole-file read that allocates. byte[] is at least usable and more accurate. In fact, it's a little dangerous to use void[], since you could assign pointer-containing values to the void[] and it should be marked as NOSCAN (no pointers inside file data). However, when using the more conventional read(void[]) makes a LOT of sense, since any T[] implicitly casts to void[]. -Steve void[] will only make sense once you've accepted that void.sizeof == 1. It is already accepted that when we talk about length in a void[], it's the number of bytes. But the data has no formal type. But any array implicitly casts to void[]. This is why it makes a good parameter for read or write (when reading or writing the binary data). Well, I guess void[] is C++'s char* for indiscriminate buffers. Speaking of which, does void* trigger strict aliasing in D? This subject seems like a hot potato no-one wants to touch. No, it's equivalent to void *, not char *. in D, ubyte[] would be the equivalent of C's char *. -Steve
std.file.read returns void[] why?
Why does the read function return void[] and not byte[] void[] read(in char[] name, size_t upTo = size_t.max);
Re: std.file.read
On 02.08.2010 5:23, bearophile wrote: Can you tell me why std.file.read() returns a void[] instead of something like a ubyte[]? Well, it magically converts to whatever array type you have. So this works: ubyte[] data = read(trash.txt); It's interesting fact deserving further investigation. It seems that void[] arrays are converted implicitly, this also works: void[] tr = malloc(20)[0..20]; data = tr; (Performing a cast(ubyte[]) in SafeD can be a problem. I presume in SafeD I have to use other safer functions to load binary data, like slurp() or something similar.) Bye, bearophile -- Dmitry Olshansky
Re: std.file.read
Dmitry Olshansky: Well, it magically converts to whatever array type you have. So this works: ubyte[] data = read(trash.txt); In more than tree years of nearly daily usage of D I have not even tried to write that code :-) Thank you, bearophile
Re: std.file.read
On 08/02/2010 10:23 AM, Dmitry Olshansky wrote: On 02.08.2010 5:23, bearophile wrote: Can you tell me why std.file.read() returns a void[] instead of something like a ubyte[]? Well, it magically converts to whatever array type you have. So this works: ubyte[] data = read(trash.txt); This code does not work for me. It's interesting fact deserving further investigation. It seems that void[] arrays are converted implicitly, this also works: void[] tr = malloc(20)[0..20]; data = tr; Neither does this. I am running 2.047, am I doing something wrong?
Re: std.file.read
Pelle: Well, it magically converts to whatever array type you have. So this works: ubyte[] data = read(trash.txt); This code does not work for me on dmd 2.047. AH, nor for me. Bye, bearophile
Re: std.file.read
On 02.08.2010 21:36, bearophile wrote: Pelle: Well, it magically converts to whatever array type you have. So this works: ubyte[] data = read(trash.txt); This code does not work for me on dmd 2.047. AH, nor for me. Bye, bearophile Hm... it doesn't ... Ouch, I was very much sure the unittest run, but unfortunately it didn't (it was some other debug output then...). Dang, DMD does not even type check the unitest code when the compiler option not passed. Again, my apologizes. -- Dmitry Olshansky
std.file.read
Can you tell me why std.file.read() returns a void[] instead of something like a ubyte[]? (Performing a cast(ubyte[]) in SafeD can be a problem. I presume in SafeD I have to use other safer functions to load binary data, like slurp() or something similar.) Bye, bearophile
