Re: Allocating a wstring on the stack (no GC)?
07.05.2014 22:26, Maxime Chevalier-Boisvert пишет: I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a regular string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance. So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated. If you have a preallocated data, just using slicing will be enough. A result will be `wstring` for immutable data or `const(wchar)[]` for non-immutable, if you respect a typesystem. If a new data is generated but you want it to be put on stack if it's small enough (you can't put big data on stack anyway) you need some allocation facility. E.g. `unstd.memory.allocation.tempAlloc` [1] will do the work. [1] http://denis-sh.bitbucket.org/unstandard/unstd.memory.allocation.html#tempAlloc -- Денис В. Шеломовский Denis V. Shelomovskij
Re: Allocating a wstring on the stack (no GC)?
On 5/7/2014 11:26 AM, Maxime Chevalier-Boisvert wrote: I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a regular string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance. So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated. Checkout std.internal.scopebuffer (new for 2.066)
Re: Allocating a wstring on the stack (no GC)?
On 5/7/2014 12:03 PM, Maxime Chevalier-Boisvert wrote: auto ptr = cast(wchar*)alloca(wchar.sizeof * len); if (ptr == null) throw new Error(...); auto mySlice = ptr[0 .. len]; Is the slice going to be allocated on the stack? (I imagine the answer is yes) Yes (slices do not copy).
Re: Allocating a wstring on the stack (no GC)?
On Wed, 07 May 2014 19:41:16 +0100, Maxime Chevalier-Boisvert
maximechevali...@gmail.com wrote:
Unless I'm misunderstanding it should be as simple as:
wchar[100] stackws; // alloca() if you need it to be dynamically sized.
A slice of this static array behaves just like a slice of a dynamic
array.
I do need it to be dynamically sized. I also want to avoid copying my
string data if possible. Basically, I just want to create a wstring
view on an existing raw buffer that exists in memory somewhere,
based on a pointer to this buffer and its length.
import std.stdio;
import core.stdc.stdlib : malloc;
import core.stdc.wchar_ : wcscpy;
wchar[] toWChar(const void *ptr, int len)
{
// Cast pointer to wchar*, create slice (on the heap?) from it (copies no
data)
return (cast(wchar*)ptr)[0..len];
}
void main()
{
// Pre-existing data
int len = 12;
wchar *ptr = cast(wchar*)malloc(len * wchar.sizeof);
wcscpy(ptr, Hello World);
// Create slice of data
wchar[] slice = toWChar(ptr, len);
writefln(%s, slice);
}
R
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Re: Allocating a wstring on the stack (no GC)?
On Wednesday, 7 May 2014 at 18:26:08 UTC, Maxime Chevalier-Boisvert wrote: I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a regular string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance. So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated. If I read your question correctly, you just want to use wstring; slices in D already work how you want. auto len = 100; auto a = new char[len]; // a (the pointer and length) is on the stack, the new memory of length len is on the GC heap auto b = a; // b is also on the stack, with the same pointer and length as a auto c = b[3 .. 40]; // c is also on the stack, with pointer equal to b.ptr + 3 and length 37
Re: Allocating a wstring on the stack (no GC)?
On Wednesday, 7 May 2014 at 18:34:10 UTC, Meta wrote: On Wednesday, 7 May 2014 at 18:29:23 UTC, Brad Anderson wrote: Unless I'm misunderstanding it should be as simple as: wchar[100] stackws; // alloca() if you need it to be dynamically sized. A slice of this static array behaves just like a slice of a dynamic array. But you should avoid slicing the static array unless it's via arr.dup. I use slicing of fixed size arrays a lot. Sure, you have to keep track of the memory to make sure it doesn't survive past the current scope (or at least isn't used past that point, depending on how security critical your application is), but that's often trivial to do and saving the extra allocation can make a big difference to performance.
Re: Allocating a wstring on the stack (no GC)?
John Colvin: Sure, you have to keep track of the memory to make sure it doesn't survive past the current scope (or at least isn't used past that point, depending on how security critical your application is), but that's often trivial to do and saving the extra allocation can make a big difference to performance. It's also bug-prone, so better to limit that usage to only the parts of the code that need the performance difference (often it's the profiler that tells where to use that). Rust language allows to perform similar things safely. Bye, bearophile
Allocating a wstring on the stack (no GC)?
I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a regular string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance. So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated.
Re: Allocating a wstring on the stack (no GC)?
On Wednesday, 7 May 2014 at 18:26:08 UTC, Maxime Chevalier-Boisvert wrote: I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a regular string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance. So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated. Unless I'm misunderstanding it should be as simple as: wchar[100] stackws; // alloca() if you need it to be dynamically sized. A slice of this static array behaves just like a slice of a dynamic array.
Re: Allocating a wstring on the stack (no GC)?
On Wednesday, 7 May 2014 at 18:29:23 UTC, Brad Anderson wrote: Unless I'm misunderstanding it should be as simple as: wchar[100] stackws; // alloca() if you need it to be dynamically sized. A slice of this static array behaves just like a slice of a dynamic array. But you should avoid slicing the static array unless it's via arr.dup.
Re: Allocating a wstring on the stack (no GC)?
Meta: But you should avoid slicing the static array unless it's via arr.dup. Slicing fixed size arrays is often necessary, because many Phobos functions don't accept fixed size arrays, they force you to lose the compile-time knowledge of the length. Bye, bearophile
Re: Allocating a wstring on the stack (no GC)?
Unless I'm misunderstanding it should be as simple as: wchar[100] stackws; // alloca() if you need it to be dynamically sized. A slice of this static array behaves just like a slice of a dynamic array. I do need it to be dynamically sized. I also want to avoid copying my string data if possible. Basically, I just want to create a wstring view on an existing raw buffer that exists in memory somewhere, based on a pointer to this buffer and its length. Side note: wouldn't alloca just produce a wchar*, not an array (wouldn't have length information)?
Re: Allocating a wstring on the stack (no GC)?
Maxime Chevalier-Boisvert: I do need it to be dynamically sized. But often you can determine statically a maximum length of the string, so you can use a fixed size stack buffer and slice it with a dynamic length. If this is not acceptable, then use alloca. Basically, I just want to create a wstring view on an existing raw buffer that exists in memory somewhere, based on a pointer to this buffer and its length. This is named slicing. You can also slice a global/static/__gshared buffer. Side note: wouldn't alloca just produce a wchar*, not an array (wouldn't have length information)? alloca returns a void*, then you can cast it to the pointer type you want, and then you slice the pointer: auto ptr = cast(wchar*)alloca(wchar.sizeof * len); if (ptr == null) throw new Error(...); auto mySlice = ptr[0 .. len]; Bye, bearophile
Re: Allocating a wstring on the stack (no GC)?
This is named slicing. You can also slice a global/static/__gshared buffer. alloca returns a void*, then you can cast it to the pointer type you want, and then you slice the pointer: auto ptr = cast(wchar*)alloca(wchar.sizeof * len); if (ptr == null) throw new Error(...); auto mySlice = ptr[0 .. len]; Is the slice going to be allocated on the stack? (I imagine the answer is yes) Thanks for your help Mr Bear.
Re: Allocating a wstring on the stack (no GC)?
Maxime Chevalier-Boisvert: Is the slice going to be allocated on the stack? (I imagine the answer is yes) Slicing doesn't change where the data is allocated. Slicing means just creating a new struct that contains a length and pointer to the data (and the struct itself is allocated in-place. So it's allocated on the stack, or inside a struct instance, or inside a class instance, or on the data segment, etc). Thanks for your help Mr Bear. You are welcome :-) Bye, bearophile
Re: Allocating a wstring on the stack (no GC)?
On Wednesday, 7 May 2014 at 18:41:17 UTC, Maxime
Chevalier-Boisvert wrote:
Basically, I just want to create a wstring view on an
existing raw buffer that exists in memory somewhere, based on
a pointer to this buffer and its length.
Looks like you actually don't need to allocate anything at all.
Any slice is just a struct with 2 fields, it does not own data
and can be used as a view to anything:
void* some_external_data;
size_t external_length;
void foo()
{
auto str =
cast(wstring[])(some_external_data[0..external_length]);
// this will create a stack instance of slice struct with
`.ptr` field pointing to same address as `some_external_data`
}
Re: Allocating a wstring on the stack (no GC)?
Is the slice going to be allocated on the stack? (I imagine the answer is yes) Slicing doesn't change where the data is allocated. Slicing means just creating a new struct that contains a length and pointer to the data (and the struct itself is allocated in-place. So it's allocated on the stack, or inside a struct instance, or inside a class instance, or on the data segment, etc). Indeed. It's the struct representing the slice I was asking about. Off to test out the performance impact :)
Re: Allocating a wstring on the stack (no GC)?
Am 07.05.2014 20:41, schrieb Maxime Chevalier-Boisvert: I do need it to be dynamically sized. I also want to avoid copying my string data if possible. Basically, I just want to create a wstring view on an existing raw buffer that exists in memory somewhere, based on a pointer to this buffer and its length. If you just need a view of the raw buffer that already exists, why don't you slice it directly? Is it neccessary that you make a copy of it? void[] rawBuffer = ...; size_t offset = ...; assert(rawBuffer.length = offset + stringLength * wchar.sizeof, out of bounds access); const(wchar)[] stringView = (cast(const(wchar)*)rawBuffer.ptr + offset)[0..stringLength]; Kind Regards Benjamin Thaut
Re: Allocating a wstring on the stack (no GC)?
On Wednesday, 7 May 2014 at 19:18:09 UTC, Maxime Chevalier-Boisvert wrote: Is the slice going to be allocated on the stack? (I imagine the answer is yes) Slicing doesn't change where the data is allocated. Slicing means just creating a new struct that contains a length and pointer to the data (and the struct itself is allocated in-place. So it's allocated on the stack, or inside a struct instance, or inside a class instance, or on the data segment, etc). Indeed. It's the struct representing the slice I was asking about. Off to test out the performance impact :) I slice is really nothing more than a fat pointer (a pointer + a size_t). You don't really allocate it any more than you allocate a pointer when you do: int* p = someInt;
Re: Allocating a wstring on the stack (no GC)?
On Wednesday, 7 May 2014 at 18:26:08 UTC, Maxime Chevalier-Boisvert wrote: I have a very specific use case (JIT compiler) in which I have a pre-allocated array of wchar string data stored somewhere in memory. I'd like to be able to create a temporary D wstring object to pass this as a regular string to other functions. For performance reasons, it would be preferable not to dynamically allocate or copy any data. Dynamically allocating the strings tends to trigger the D GC which severely impacts the performance. So, my question is, would it be possible for me to allocate a wstring object on the stack, and manually set its string data pointer and length? If so, how? Your wizardly help is much appreciated. As mentioned a slice is the stack allocated view which you desire, however your selection of terms, wchar string and wstring suggest to me that you'll also be dealing with a conversion from mutable to immutable. To put my expectations of these terms into types: wchar string : wchar[] wstring : immutable(wchar)[] If I have understood this correctly then you'll also want to look toward std.exception.assumeUnique() which you can call on a slice (wchar[]). However, you should not use this if you will maintain a mutable reference to your data.
