Re: may gc free malloced memory?
On Thursday, 12 September 2013 at 05:36:31 UTC, Alexandr Druzhinin wrote: Some C function malloc-ed memory. This memory should be freeed much later. I don't want to manually call C function to free this memory in some point later, so may I in some way ask gc to free this memory using something like addRoot(for instance) or else or the true way is to copy malloc-ed memory to gc-allocated memory and free malloc-ed memory at once? Like: ubyte data* = cfunction_allocates_memory(); auto gcmemory = data[0..length(data)]; cfunction_frees_memory(data); // work with gcmemory only or ubyte data* = cfunction_allocates_memory(); GC.someUnknownToMeFunction(data); // now gc will control this memory No. Only free can be used with malloc. The memory comes from distinct pools. Another option could be to use GC.malloc, and memcpy your old mmory into your new memory, free the old memory, and use your new block. GC.malloc, as the name suggests, is a malloc, but done by the GC.
Re: Greedy memory handling
On 2013-09-11 10:06, monarch_dodra wrote: I have a function that will *massively* benefit from having a persistent internal buffer it can re-use (and grow) from call to call, instead of re-allocating on every call. What I don't want is either of: 1. To set a fixed limitation of size, if the user ends up making repeated calls to something larger to my fixed size. 2. For a single big call which will allocate a HUGE internal buffer that will consume all my memory. What I need is some sort of lazy buffer. Basically, the allocation holds, but I don't want the to prevent the GC from collecting it if it deems it has gotten too big, or needs more memory. Any idea on how to do something like that? Or literature? How about keeping a stack or static buffer. If that gets too small use a new buffer. When you're done with the new buffer set it to null to allow the GC to collect it. Then repeat. -- /Jacob Carlborg
Re: may gc free malloced memory?
On Thursday, 12 September 2013 at 05:59:33 UTC, monarch_dodra wrote: On Thursday, 12 September 2013 at 05:36:31 UTC, Alexandr Druzhinin wrote: Some C function malloc-ed memory. This memory should be freeed much later. I don't want to manually call C function to free this memory in some point later, so may I in some way ask gc to free this memory using something like addRoot(for instance) or else or the true way is to copy malloc-ed memory to gc-allocated memory and free malloc-ed memory at once? Like: ubyte data* = cfunction_allocates_memory(); auto gcmemory = data[0..length(data)]; cfunction_frees_memory(data); // work with gcmemory only or ubyte data* = cfunction_allocates_memory(); GC.someUnknownToMeFunction(data); // now gc will control this memory No. Only free can be used with malloc. The memory comes from distinct pools. Another option could be to use GC.malloc, and memcpy your old mmory into your new memory, free the old memory, and use your new block. GC.malloc, as the name suggests, is a malloc, but done by the GC. You could also use some kind of helper class. Perhaps Phobos has a facility for this, but to illustrate the idea: http://dpaste.dzfl.pl/805a61c0 However note that the memory isn't guaranteed to be freed this way. Only if the GC heap gets full and the collector runs.
Re: may gc free malloced memory?
12.09.2013 14:45, Rene Zwanenburg пишет: On Thursday, 12 September 2013 at 05:59:33 UTC, monarch_dodra wrote: On Thursday, 12 September 2013 at 05:36:31 UTC, Alexandr Druzhinin wrote: Some C function malloc-ed memory. This memory should be freeed much later. I don't want to manually call C function to free this memory in some point later, so may I in some way ask gc to free this memory using something like addRoot(for instance) or else or the true way is to copy malloc-ed memory to gc-allocated memory and free malloc-ed memory at once? Like: ubyte data* = cfunction_allocates_memory(); auto gcmemory = data[0..length(data)]; cfunction_frees_memory(data); // work with gcmemory only or ubyte data* = cfunction_allocates_memory(); GC.someUnknownToMeFunction(data); // now gc will control this memory No. Only free can be used with malloc. The memory comes from distinct pools. Another option could be to use GC.malloc, and memcpy your old mmory into your new memory, free the old memory, and use your new block. GC.malloc, as the name suggests, is a malloc, but done by the GC. You could also use some kind of helper class. Perhaps Phobos has a facility for this, but to illustrate the idea: http://dpaste.dzfl.pl/805a61c0 However note that the memory isn't guaranteed to be freed this way. Only if the GC heap gets full and the collector runs. Ok. I just think that copying from one pool to another is excessive and may be there is a some way to avoid it. Thanks for the answers!
Re: VisualD import
On Wednesday, 11 September 2013 at 22:15:07 UTC, Rainer Schuetze
wrote:
On 11.09.2013 23:42, Lemonfiend wrote:
On Wednesday, 11 September 2013 at 20:36:39 UTC, Rainer
Schuetze wrote:
On 11.09.2013 18:13, Lemonfiend wrote:
Oops, I forgot to say what I actually did.
I added derelict to Compiler-General-Additional Imports.
The code is just this:
module main;
import std.stdio;
import derelict.opengl3.gl3;
void main()
{
writeln(Hello D-World!);
}
And the build output is a symbol undefined linker issue:
-- Rebuild All started: Project: Test, Configuration:
Debug Win32
--
Building Debug\Test.exe...
OPTLINK (R) for Win32 Release 8.00.12
Copyright (C) Digital Mars 1989-2010 All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
Debug\Test.obj(Test)
Error 42: Symbol Undefined
_D8derelict7opengl33gl312__ModuleInfoZ
Building Debug\Test.exe failed!
Details saved as
file://C:\D\Test\Test\Debug\Test.buildlog.html
Build time: 3 s
Solution build stopped.
Build has been canceled.
Did you add the derelict library/libraries as linker inputs?
I haven't compiled derelict to a lib, I'm using the source
directly.
With rdmd I can simply do -Ipath\to\derelict\source
The compilation model of rdmd is not supported by Visual D. You
might be able to use rdmd as other compiler in the project
options with additional options --build-only.
On the other hand, you can also create a library from the
project templates, then drag the source folder into the project
to add all files. Then set a project dependency of your
application to the library.
You can also create a dub package and generate visuald project
from dub.
override and package
Code:
import std.stdio;
class T1 {
protected:
void _apply() {
writeln(Call T1);
}
}
class T2 : T1 {
public:
override void _apply() {
writeln(Call T2);
}
}
class T3 : T1 {
protected:
override void _apply() {
writeln(Call T3);
}
}
class T4 : T1 {
package:
void _apply() { /// -- [1]
writeln(Call T4);
}
}
void main()
{
T1 t1 = new T1();
T2 t2 = new T2();
T3 t3 = new T3();
T4 t4 = new T4();
t1._apply();
t2._apply();
t3._apply();
t4._apply();
}
Produce the correct output:
Call T1
Call T2
Call T3
Call T4
If I remove 'override' from T3 (or also T2) I get the correct
deprecation message:
/d172/f194.d(19): Deprecation: overriding base class function
without using override attribute is deprecated (f194.T3._apply
overrides f194.T1._apply)
But if I try to write 'override' before [1], I get this error
message:
Error: function T4._apply cannot override a non-virtual function
This seems inconsistent. I really overwrite the method, and then
I put it in a package label.
Re: override and package
Same with private, of course.
Re: Adding libraries to an executable
On Thursday, 12 September 2013 at 01:16:28 UTC, Mike Parker wrote: On 9/12/2013 6:20 AM, Anton Alexeev wrote: On Tuesday, 10 September 2013 at 11:06:04 UTC, Jacob Carlborg wrote: On 2013-09-10 11:04, Anton Alexeev wrote: Can be but not libphobos2 Link statically with it, which is does by default. Just compile with: dmd -L-lcurl test.d The point is: http://pastebin.com/0VkYgFix Apparently you need to install the libcurl dev package. The linker can't find libcurl.a. I've installed libcurl4-openssl-dev 7.29.0-1ubuntu3.1 before I asked for help here. Else I can't even compile with that command: dmd -L-lphobos2 -L-lcurl test.d
Re: override and package
On 2013-09-12 11:28, Namespace wrote:
But if I try to write 'override' before [1], I get this error message:
Error: function T4._apply cannot override a non-virtual function
This seems inconsistent. I really overwrite the method, and then I put
it in a package label.
I think the error message is pretty clear. You cannot override a
function that isn't virtual. Private and package methods are not
virtual. Example:
class Base
{
void foo ()
{
writeln(Base.foo);
}
}
class Sub : Base
{
package void foo ()
{
writeln(Sub.foo);
}
}
void main ()
{
auto sub = new Sub;
sub.foo(); // prints Sub.foo as expected
Base base = sub;
base.foo(); // prints Base.foo
}
--
/Jacob Carlborg
Re: Adding libraries to an executable
On 2013-09-12 12:14, Anton Alexeev wrote: I've installed libcurl4-openssl-dev 7.29.0-1ubuntu3.1 before I asked for help here. Else I can't even compile with that command: dmd -L-lphobos2 -L-lcurl test.d How does your dmd.conf file look like? -- /Jacob Carlborg
Re: override and package
On Thursday, 12 September 2013 at 11:29:22 UTC, Jacob Carlborg
wrote:
On 2013-09-12 11:28, Namespace wrote:
But if I try to write 'override' before [1], I get this error
message:
Error: function T4._apply cannot override a non-virtual
function
This seems inconsistent. I really overwrite the method, and
then I put
it in a package label.
I think the error message is pretty clear. You cannot override
a function that isn't virtual. Private and package methods are
not virtual. Example:
class Base
{
void foo ()
{
writeln(Base.foo);
}
}
class Sub : Base
{
package void foo ()
{
writeln(Sub.foo);
}
}
void main ()
{
auto sub = new Sub;
sub.foo(); // prints Sub.foo as expected
Base base = sub;
base.foo(); // prints Base.foo
}
Obvious. But what happend? Is the original _apply hidden?
Re: override and package
On 2013-09-12 13:34, Namespace wrote: Obvious. But what happend? Is the original _apply hidden? If you call through an object of the same type, T4 in this case, the method in the base class is hidden. If you call through a base class reference, T1 in this case, the _apply method in the subclass is hidden. -- /Jacob Carlborg
Re: override and package
On 2013-09-12 13:39, Jacob Carlborg wrote: If you call through an object of the same type, T4 in this case, the method in the base class is hidden. If you call through a base class reference, T1 in this case, the _apply method in the subclass is hidden. I'm guessing it's the same as overriding non-virtual methods in C++, if you're familiar with that. -- /Jacob Carlborg
Re: __FILE__ and __LINE__ again...
On Tuesday, 10 September 2013 at 16:58:54 UTC, H. S. Teoh wrote: On Tue, Sep 10, 2013 at 06:50:03PM +0200, Dicebot wrote: On Tuesday, 10 September 2013 at 16:45:33 UTC, H. S. Teoh wrote: but you can get rid of this with link-time optimization (on Posix, you'd add -L-gc-sections to your dmd command-line: this will cause ld to delete code sections that are never referenced, which includes the log() instantiations if indeed they have been inlined). Using `--gc-sections` requires each function and data item to be placed into its own section (-fdata-sections and -ffunction-sections in gcc). AFAIK dmd does not do it. Hmm. Does that mean gdc supports that, then? Actually I was wrong, looks like DMD does section differentiation by default. I am wondering if --gc-sections should be made default on Linux targets...
Re: Greedy memory handling
On Thu, Sep 12, 2013 at 08:27:59AM +0200, Jacob Carlborg wrote:
On 2013-09-11 10:06, monarch_dodra wrote:
I have a function that will *massively* benefit from having a
persistent internal buffer it can re-use (and grow) from call to
call, instead of re-allocating on every call.
What I don't want is either of:
1. To set a fixed limitation of size, if the user ends up making
repeated calls to something larger to my fixed size.
2. For a single big call which will allocate a HUGE internal buffer
that will consume all my memory.
What I need is some sort of lazy buffer. Basically, the allocation
holds, but I don't want the to prevent the GC from collecting it if
it deems it has gotten too big, or needs more memory.
Any idea on how to do something like that? Or literature?
How about keeping a stack or static buffer. If that gets too small
use a new buffer. When you're done with the new buffer set it to
null to allow the GC to collect it. Then repeat.
[...]
The problem is, he wants to reuse the buffer next time if the GC hasn't
collected it yet.
Here's an idea, though. It doesn't completely solve the problem, but it
just occurred to me that weak pointers (i.e., ignored by the GC for
the purposes of marking) can be simulated by XOR'ing the pointer value
with some mask so that it's not recognized as a pointer by the GC. This
can be encapsulated by a weak pointer struct that automatically does the
translation:
struct WeakPointer(T) {
enum size_t mask = 0xdeadbeef;
union Impl {
T* ptr;
size_t uintVal;
}
Impl impl;
void set(T* ptr) @system {
impl.ptr = ptr;
impl.uintVal ^= mask;
}
T* get() @system {
Impl i = impl;
i.uintVal ^= mask;
return i.ptr;
}
}
WeakPointer!Buffer bufferRef;
void doWork(Args...) {
T* buffer;
if (bufferRef.get() is null) {
// Buffer hasn't been allocated yet
buffer = allocateNewBuffer();
bufferRef.set(buffer);
} else {
void *p;
core.memory.GC.getAttr(p);
if (p is null || p != bufferRef.get()) {
// GC has collected previous buffer
buffer = allocateNewBuffer();
bufferRef.set(buffer);
}
}
useBuffer(buffer);
...
}
Note that the inner if block is not 100% safe, because there's no
guarantee that even if the base pointer of the block hasn't changed, the
GC hasn't reallocated the block to somebody else. So this part is still
yet to be solved.
T
--
It is widely believed that reinventing the wheel is a waste of time; but
I disagree: without wheel reinventers, we would be still be stuck with
wooden horse-cart wheels.
Re: __FILE__ and __LINE__ again...
On Thu, Sep 12, 2013 at 03:37:58PM +0200, Dicebot wrote: On Tuesday, 10 September 2013 at 16:58:54 UTC, H. S. Teoh wrote: On Tue, Sep 10, 2013 at 06:50:03PM +0200, Dicebot wrote: On Tuesday, 10 September 2013 at 16:45:33 UTC, H. S. Teoh wrote: but you can get rid of this with link-time optimization (on Posix, you'd add -L-gc-sections to your dmd command-line: this will cause ld to delete code sections that are never referenced, which includes the log() instantiations if indeed they have been inlined). Using `--gc-sections` requires each function and data item to be placed into its own section (-fdata-sections and -ffunction-sections in gcc). AFAIK dmd does not do it. Hmm. Does that mean gdc supports that, then? Actually I was wrong, looks like DMD does section differentiation by default. I am wondering if --gc-sections should be made default on Linux targets... Interesting. However, I ran into some runtime segfaults caused by --gc-sections yesterday. I didn't investigate further, but that makes me hesitant to make --gc-sections the default. Something, somewhere, is being broken by --gc-sections. It doesn't happen in all cases, though. Only some of my programs are affected by it. I should investigate this when I get some time. T -- Let's call it an accidental feature. -- Larry Wall
Re: __FILE__ and __LINE__ again...
On Thursday, 12 September 2013 at 13:58:03 UTC, H. S. Teoh wrote: Interesting. However, I ran into some runtime segfaults caused by --gc-sections yesterday. I didn't investigate further, but that makes me hesitant to make --gc-sections the default. Something, somewhere, is being broken by --gc-sections. It doesn't happen in all cases, though. Only some of my programs are affected by it. I should investigate this when I get some time. Actually, nevermind, dmd only does it for small portion of the code :) However, I have tested -ffunction-sections + -fdata-sections + --gc-sections on gdc and have reduced binary size _twice_ for a simple `writeln(arr.map(a = a*2)())` snippet. Which is only partially cool because gdc binary is more than twice larger than dmd one for same code :D Problem with --gc-sections is that it can't be used when building libraries and any with executable that expose parts of own code to shared libraries. It is essentially a whole program optimization and it can't be done without compiler support in cases where binary is _not_ the whole program.
Re: Adding libraries to an executable
On Thursday, 12 September 2013 at 11:30:05 UTC, Jacob Carlborg wrote: On 2013-09-12 12:14, Anton Alexeev wrote: I've installed libcurl4-openssl-dev 7.29.0-1ubuntu3.1 before I asked for help here. Else I can't even compile with that command: dmd -L-lphobos2 -L-lcurl test.d How does your dmd.conf file look like? ; ; dmd.conf file for dmd ; ; dmd will look for dmd.conf in the following sequence of directories: ; - current working directory ; - directory specified by the HOME environment variable ; - directory dmd resides in ; - /etc directory ; ; Names enclosed by %% are searched for in the existing environment and inserted ; ; The special name %@P% is replaced with the path to this file ; [Environment] DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import -L-L/usr/lib/i386-linux-gnu -L-L/usr/lib/x86_64-linux-gnu -L--no-warn-search-mismatch -L--export-dynamic
Handling exceptions from Windows DLL
Hello. There was an issue 2 years ago: https://github.com/D-Programming-Language/druntime/pull/92 It seems that this bug is still here (I can't handle D Exception from dll builded with dmd and dmc link). Will this bug get fixed? Do other D compilers have the same issue?
Re: override and package
On Thursday, 12 September 2013 at 11:29:22 UTC, Jacob Carlborg wrote: On 2013-09-12 11:28, Namespace wrote: But if I try to write 'override' before [1], I get this error message: Error: function T4._apply cannot override a non-virtual function This seems inconsistent. I really overwrite the method, and then I put it in a package label. I think the error message is pretty clear. You cannot override a function that isn't virtual. Private and package methods are not virtual. Actually error message is misleading - it complains as T1._apply would be non-virtual function which is wrong, it is one. The problem is not in case of overriding non-virtual function but in attempt to override virtual function (private) by non-virtual one (package). I think error mesage should be something like: Error: function T4._apply is non-virtual function and cannot ovverride virtual function T1._apply
Re: Greedy memory handling
12-Sep-2013 17:51, H. S. Teoh пишет:
On Thu, Sep 12, 2013 at 08:27:59AM +0200, Jacob Carlborg wrote:
On 2013-09-11 10:06, monarch_dodra wrote:
I have a function that will *massively* benefit from having a
persistent internal buffer it can re-use (and grow) from call to
call, instead of re-allocating on every call.
What I don't want is either of:
1. To set a fixed limitation of size, if the user ends up making
repeated calls to something larger to my fixed size.
2. For a single big call which will allocate a HUGE internal buffer
that will consume all my memory.
What I need is some sort of lazy buffer. Basically, the allocation
holds, but I don't want the to prevent the GC from collecting it if
it deems it has gotten too big, or needs more memory.
Any idea on how to do something like that? Or literature?
How about keeping a stack or static buffer. If that gets too small
use a new buffer. When you're done with the new buffer set it to
null to allow the GC to collect it. Then repeat.
[...]
The problem is, he wants to reuse the buffer next time if the GC hasn't
collected it yet.
Here's an idea, though. It doesn't completely solve the problem, but it
just occurred to me that weak pointers (i.e., ignored by the GC for
the purposes of marking) can be simulated by XOR'ing the pointer value
with some mask so that it's not recognized as a pointer by the GC. This
can be encapsulated by a weak pointer struct that automatically does the
translation:
struct WeakPointer(T) {
enum size_t mask = 0xdeadbeef;
union Impl {
T* ptr;
size_t uintVal;
}
Impl impl;
void set(T* ptr) @system {
impl.ptr = ptr;
impl.uintVal ^= mask;
}
T* get() @system {
Impl i = impl;
i.uintVal ^= mask;
return i.ptr;
}
}
WeakPointer!Buffer bufferRef;
void doWork(Args...) {
T* buffer;
if (bufferRef.get() is null) {
// Buffer hasn't been allocated yet
buffer = allocateNewBuffer();
bufferRef.set(buffer);
} else {
void *p;
core.memory.GC.getAttr(p);
This line above is not 100% good idea .. at least with deadbeaf as mask.
If we do know what OS you compile for we may just flip the say upper bit
and get a pointer into kernel space (and surely that isn't in GC pool).
Even then your last paragraph pretty much destroys it.
Better option is to have finalizer hooked up to set some flag. Then
_after_ restoring the pointer we consult that flag variable.
if (p is null || p != bufferRef.get()) {
// GC has collected previous buffer
buffer = allocateNewBuffer();
bufferRef.set(buffer);
}
}
useBuffer(buffer);
...
}
Note that the inner if block is not 100% safe, because there's no
guarantee that even if the base pointer of the block hasn't changed, the
GC hasn't reallocated the block to somebody else. So this part is still
yet to be solved.
T
--
Dmitry Olshansky
Re: Any idea for a solution to handle overloads when dynamically implementing methods?
On Wednesday, 11 September 2013 at 18:24:31 UTC, H. S. Teoh wrote:
On Mon, Sep 09, 2013 at 10:16:42PM +0200, Gary Willoughby wrote:
Just wondered if i could pick you brains for a nice solution to
dynamically add methods to a class, paying particular
attention to
overloads. I'm currently writing a mocking framework and
everything's coming along nicely and i'm wondering how to
handle
replacing overloads of the mocked class.
To create a new mocked class this is the code:
auto mock = new Mock!Person();
Simple enough, mock now contains an extended class with all the
methods set to assert(false) because there are no
implementations
yet. What i need to do is to add the implementations
dynamically.
This is the code i propose.
mock.addMethod(getAge, int delegate(){
return 40;
});
assert(mock.getAge() == 40);
Which i guess would be easy to implement but it doesn't handle
overloads because the method string doesn't contain enough
information to define which overload it's implementing.
Any nice ideas what would be a nice way of supporting this? I
thought i'd ask while i have a think and get some tea. :)
One idea I have is to use the built-in typetuples as a way of
disambiguating between different overloads. For example,
something like
this:
// This captures the function argument type list in a form that
// we can call .mangleof on.
template ArgTypesWrapper(ArgTypes...) { }
// This builds a unique string to identify a specific overload
// based on the function name and the .mangleof of its argument
// types. The key to this trick is that the .mangleof of a
// template encodes its argument types, so it is unique per
// combination of argument types.
template FuncSignature(string funcName, ArgTypes...) {
enum FuncSignature = funcName ~ ArgTypesWrapper.mangleof;
}
class Mock(... /* whatever you currently have here */) {
// This unfortunately has to be a template function in
// order to be able to capture the argument types of the
// delegate in the typetuple A. This may complicate the
// implementation of how you'd actually dispatch to the
// overload implementation at runtime.
void addMethod(R, A...)(string funcName, R delegate(A...) dg)
{
string overloadName = FuncSignature!(funcName, A);
// Now overloadName should be a unique string
// representing that particular combination of
// function name and argument types, i.e., it's
// a function signature. So you can use it to
// identify which overload is which.
}
}
T
Thanks for the idea, i have implemented something based on this
and it seems to be working well.
What does __parameters return?
The documentation says the parameter tuple of a function, delegate, or function pointer. This includes the parameter types, names, and default values. .. but what type is it? Error: argument (int i, char c, string s, bool b = false) to typeof is not an expression How am I supposed to get the parameter names using this? It's possible to use .stringof, and then regular string handling to extract the names, but I guess there should be a simpler way if only I knew what it returned.
Re: Safety/purity and assert/enforce error messages
On Thu, Sep 12, 2013 at 09:12:18PM +0200, bearophile wrote:
H. S. Teoh:
In phobos git HEAD, std.format has been made pure @safe nothrow (and
CTFE-able), so you should be able to write your assert as:
assert(condition, format(x = %s, blahblah, x));
With the latest DMD from updated GIT head:
import std.string: format;
void main() pure nothrow {
string x = hello;
bool condition = true;
assert(condition, format(x = %s, blahblah, x));
}
It gives:
test.d(5): Error: 'std.string.format!(char, string).format' is not nothrow
test.d(2): Error: function 'D main' is nothrow yet may throw
[...]
Oops. Apparently I wrote nothrow but forgot to test it. I did test pure
and @safe, though, so at least those two should work.
T
--
Food and laptops don't mix.
Re: cascade operator or nearest equivalent
Daniel Davidson:
I am using Dart for code generation but would like to consider
D if I can find a convenient replacement for the following
declarative style:
Replacing Dart with D seems quite strange, such two languages
have so much different usage niches.
var dateRange = struct('date_range')
..doc = 'Basic pair of start and end dates'
..unitTest = true
..publicSection = true
..members = [
member('start_date')
..type = 'Date',
member('end_date')
..type = 'Date',
];
A similar syntax is not allowed in D, but there are two things
that help for this: the C-style initialization of structs, that
support field names too, and the with(){} statement.
Bye,
bearophile
Re: Linking Trouble (Linux)
On Thursday, 12 September 2013 at 19:46:21 UTC, Craig Dillabaugh wrote: I just upgraded my Linux distro (openSuse) and now when trying to compile a project with dub I am getting linking errors. Compilation goes OK until the linking stage, when I get the following: Linking... dmd -of/home/craig/cloud/vibe-tiles/vibe /home/craig/cloud/vibe-tiles/temp.o -L-levent_pthreads -L-levent -L-lssl -L-lcrypto -g clip Strangely, libdl.so.2 and libz.so.1 are both there and libcrypto is linked to them. clip Craig OK, so I solved my compilation problem by passing the following as the linker command (added -L-ldl and -L-lz): dmd -of/home/craig/cloud/vibe-tiles/vibe /home/craig/cloud/vibe-tiles/temp.o -L-levent_pthreads -L-levent -L-lssl -L-ldl -L-lz -L-lcrypto -g How the GCC linker decides which libraries I need to explicitly specify is indeed a mystery to me. Craig
Re: Safety/purity and assert/enforce error messages
On Thursday, September 12, 2013 13:21:26 H. S. Teoh wrote:
On Thu, Sep 12, 2013 at 04:07:24PM -0400, Jonathan M Davis wrote:
On Thursday, September 12, 2013 12:42:34 H. S. Teoh wrote:
On Thu, Sep 12, 2013 at 09:12:18PM +0200, bearophile wrote:
H. S. Teoh:
In phobos git HEAD, std.format has been made pure @safe nothrow
(and CTFE-able), so you should be able to write your assert as:
assert(condition, format(x = %s, blahblah, x));
With the latest DMD from updated GIT head:
import std.string: format;
void main() pure nothrow {
string x = hello;
bool condition = true;
assert(condition, format(x = %s, blahblah, x));
}
It gives:
test.d(5): Error: 'std.string.format!(char, string).format' is not
nothrow
test.d(2): Error: function 'D main' is nothrow yet may throw
[...]
Oops. Apparently I wrote nothrow but forgot to test it. I did test
pure and @safe, though, so at least those two should work.
format can't be nothrow, because it throws when you screw up the
format specifiers. You have to wrap it in a try-catch block and
assert(0) in the catch block if you want to put it in a nothrow
function. std.datetime does this in at least a few places.
[...]
Right. Makes me wanna suggest adding this template to std.exception:
auto assumeWontThrow(alias fun, T...)(T args) nothrow {
try {
return fun(args);
} catch(Exception) {
// You broke your promise, I die.
assert(0);
}
}
Then you can write things like:
int mayThrow(int arg) {
if (arg 0) throw new Exception(...);
return BBN(arg);
}
int iWontThrow() nothrow { // -- N.B. this function is nothrow
static assert(123 0);
return assumeWontThrow!mayThrow(123);
}
Sounds like a good suggestion, though an actual implementation should have an
error message in the assert(0). You could also implement it as a lazy
parameter like enforce and assertNotThrown do (which I think is more user-
friendly), but that would likely be less efficient due to how lazy doesn't get
optimized very well (or at least, it's my understanding that it doesn't).
- Jonathan M Davis
Re: Safety/purity and assert/enforce error messages
On Thu, Sep 12, 2013 at 10:17:30PM +0200, Joseph Rushton Wakeling wrote:
On 12/09/13 22:07, Jonathan M Davis wrote:
format can't be nothrow, because it throws when you screw up the
format specifiers. You have to wrap it in a try-catch block and
assert(0) in the catch block if you want to put it in a nothrow
function. std.datetime does this in at least a few places.
The annoyance of this is that it means that any otherwise @safe pure
nothrow function that has an assert() in it can be blocked from being
nothrow if it needs a formatted string for the assert failure message.
Note that this restriction still applies even if the code is being
compiled with -release and the assertion therefore stripped out.
Or am I missing a trick as to how to deal with assert() and enforce()
... ? :-)
Using assumeWontThrow, as defined in my previous post, you can write
this:
auto myFunc(int x) nothrow {
// Look, ma! This line doesn't violate nothrow!
assert(isValid(x), assumeWontThrow!format(
Invalid args (%s), dude!, x));
... // do work here
}
:-)
T
--
Uhh, I'm still not here. -- KD, while away on ICQ.
Cannot access frame of insert whatever here
I haven't done any D coding since May, but picked up my little
toy code again now.
Last time I had a lot of hassle with not being able to
store/access symbols through templates, and I got the same thing
now.
Am I the only one hitting these all the time?
I'm basically playing around with UDAs:
@notNull class A {
@matches([0-9]+) string s;
@interval![)(0, 10) int i;
}
void f(A a) {
mixin enforceValidParameters;
enforceValidParameters();
}
f(null); // Throws because a is null
f(new A()); // Throws because a.s doesn't match and i isn't
within [0,10)
I need a mixin there because I haven't found another way of
getting the correct function without passing any parameters. And
the second call because the mixin cannot call any code itself,
only declare.
.. But the above doesn't work. I have to do this instead:
void f(A a, A b) {
validate!(a, b)();
}
which doesn't scale. It's bad enough that I need to run a
function after mixing in code.
.. Just a bit of a rant - unable to access frames sucks :/
Re: Greedy memory handling
12-Sep-2013 20:51, H. S. Teoh пишет:
On Thu, Sep 12, 2013 at 07:50:25PM +0400, Dmitry Olshansky wrote:
12-Sep-2013 17:51, H. S. Teoh пишет:
[...]
struct WeakPointer(T) {
enum size_t mask = 0xdeadbeef;
union Impl {
T* ptr;
size_t uintVal;
}
Impl impl;
void set(T* ptr) @system {
impl.ptr = ptr;
impl.uintVal ^= mask;
}
T* get() @system {
Impl i = impl;
i.uintVal ^= mask;
return i.ptr;
}
}
WeakPointer!Buffer bufferRef;
void doWork(Args...) {
T* buffer;
if (bufferRef.get() is null) {
// Buffer hasn't been allocated yet
buffer = allocateNewBuffer();
bufferRef.set(buffer);
} else {
void *p;
core.memory.GC.getAttr(p);
This line above is not 100% good idea .. at least with deadbeaf as
mask.
If we do know what OS you compile for we may just flip the say upper
bit and get a pointer into kernel space (and surely that isn't in GC
pool). Even then your last paragraph pretty much destroys it.
Well, that was just an example value. :) If we know which OS it is and
how it assigns VM addresses, then we can adjust the mask appropriately.
But yeah, calling GC.getAttr is unreliable since you can't tell whether
the block is what you had before, or somebody else's new data.
It occured to me that there are modes where full address space is
available, typically so on x86 app running on top of x64 kernel (e.g. in
Windows Wow64 could do that, Linux also has so-called x32 ABI).
[...]
Better option is to have finalizer hooked up to set some flag. Then
_after_ restoring the pointer we consult that flag variable.
Good idea. The problem is, how to set a finalizer on a memory block that
can change in size? The OP's original situation was that the buffer can
be extended while in use, but I don't know of any D type that can
associate a dtor with a ubyte[] array (note that the GC collecting the
wrapper struct/class around the ubyte[] is not the same as collecting
the actual memory block storing the ubyte[] -- the former can happen
without the latter).
Double indirection? Allocate a class that has finalizer, hold that via
weak-ref. The wrapper in turn contains a pointer to the buffer. The
interesting point then is that one may allocate said buffer via C's realloc.
Then once helper struct is collected the finalizer is called and this is
where we call free to cleanup C's heap.
I'm thinking this actually is going to work.
--
Dmitry Olshansky
Re: Safety/purity and assert/enforce error messages
H. S. Teoh:
In phobos git HEAD, std.format has been made pure @safe nothrow
(and
CTFE-able), so you should be able to write your assert as:
assert(condition, format(x = %s, blahblah, x));
With the latest DMD from updated GIT head:
import std.string: format;
void main() pure nothrow {
string x = hello;
bool condition = true;
assert(condition, format(x = %s, blahblah, x));
}
It gives:
test.d(5): Error: 'std.string.format!(char, string).format' is
not nothrow
test.d(2): Error: function 'D main' is nothrow yet may throw
Bye,
bearophile
Re: Safety/purity and assert/enforce error messages
On Thursday, September 12, 2013 12:42:34 H. S. Teoh wrote:
On Thu, Sep 12, 2013 at 09:12:18PM +0200, bearophile wrote:
H. S. Teoh:
In phobos git HEAD, std.format has been made pure @safe nothrow (and
CTFE-able), so you should be able to write your assert as:
assert(condition, format(x = %s, blahblah, x));
With the latest DMD from updated GIT head:
import std.string: format;
void main() pure nothrow {
string x = hello;
bool condition = true;
assert(condition, format(x = %s, blahblah, x));
}
It gives:
test.d(5): Error: 'std.string.format!(char, string).format' is not nothrow
test.d(2): Error: function 'D main' is nothrow yet may throw
[...]
Oops. Apparently I wrote nothrow but forgot to test it. I did test pure
and @safe, though, so at least those two should work.
format can't be nothrow, because it throws when you screw up the format
specifiers. You have to wrap it in a try-catch block and assert(0) in the catch
block if you want to put it in a nothrow function. std.datetime does this in
at least a few places.
- Jonathan M Davis
Re: Safety/purity and assert/enforce error messages
On Thu, Sep 12, 2013 at 04:07:24PM -0400, Jonathan M Davis wrote:
On Thursday, September 12, 2013 12:42:34 H. S. Teoh wrote:
On Thu, Sep 12, 2013 at 09:12:18PM +0200, bearophile wrote:
H. S. Teoh:
In phobos git HEAD, std.format has been made pure @safe nothrow
(and CTFE-able), so you should be able to write your assert as:
assert(condition, format(x = %s, blahblah, x));
With the latest DMD from updated GIT head:
import std.string: format;
void main() pure nothrow {
string x = hello;
bool condition = true;
assert(condition, format(x = %s, blahblah, x));
}
It gives:
test.d(5): Error: 'std.string.format!(char, string).format' is not nothrow
test.d(2): Error: function 'D main' is nothrow yet may throw
[...]
Oops. Apparently I wrote nothrow but forgot to test it. I did test
pure and @safe, though, so at least those two should work.
format can't be nothrow, because it throws when you screw up the
format specifiers. You have to wrap it in a try-catch block and
assert(0) in the catch block if you want to put it in a nothrow
function. std.datetime does this in at least a few places.
[...]
Right. Makes me wanna suggest adding this template to std.exception:
auto assumeWontThrow(alias fun, T...)(T args) nothrow {
try {
return fun(args);
} catch(Exception) {
// You broke your promise, I die.
assert(0);
}
}
Then you can write things like:
int mayThrow(int arg) {
if (arg 0) throw new Exception(...);
return BBN(arg);
}
int iWontThrow() nothrow { // -- N.B. this function is nothrow
static assert(123 0);
return assumeWontThrow!mayThrow(123);
}
T
--
It won't be covered in the book. The source code has to be useful for
something, after all. -- Larry Wall
Re: Greedy memory handling
On Thursday, 12 September 2013 at 19:13:40 UTC, Dmitry Olshansky
wrote:
Double indirection? Allocate a class that has finalizer, hold
that via weak-ref. The wrapper in turn contains a pointer to
the buffer. The interesting point then is that one may allocate
said buffer via C's realloc.
Then once helper struct is collected the finalizer is called
and this is where we call free to cleanup C's heap.
I'm thinking this actually is going to work.
Yum. I like this.
I was going to say: At the end of the day, if the GC doesn't
*tell* us the collection happened, then the problem is not
solve-able. We'd need a way that would allow the GC to tell us
the memory was *finalized*. And then I'd go on to say since our
GC is non-finalizing, there is simply no solution.
But then classes. Derp.
I'd be real interested in having a finalized solution. The
details of how memory addressing is not my strong suite, so I
wouldn't trust myself with all those union{ptr/size_t} things.
Thanks, I'll start toying around with this :)
Re: What does __parameters return?
On Thu, Sep 12, 2013 at 08:44:29PM +0200, simendsjo wrote:
The documentation says the parameter tuple of a function, delegate,
or function pointer. This includes the parameter types, names, and
default values.
.. but what type is it?
Error: argument (int i, char c, string s, bool b = false) to typeof
is not an expression
How am I supposed to get the parameter names using this?
It's possible to use .stringof, and then regular string handling to
extract the names, but I guess there should be a simpler way if only
I knew what it returned.
You're not going to like the answer to this one... :)
Here's some example code that uses __parameters to inspect a function
and print out a list of its parameter types and names (my explanations
are in the comments):
-snip-
import std.stdio;
// Example function we want to examine parameters for
int func(string x, float y, int z=123) {
return 0;
}
void main() {
// __parameters appears to only work inside is(...), in a quirky
// special-cased syntax that ONLY works when written exactly in the
// form is(TypeOfFunc Ident == __parameters), in which case Ident gets
// aliased to a parameter tuple, an elusive compiler internal entity
// that is similar to, but not the same as, a type tuple.
static if (is(typeof(func) Params == __parameters)) {
// You already know what does one this, I believe.
pragma(msg, Params.stringof);
// foreach works on parameter tuples, but it acts very weird.
// If a single argument is given, it gets assigned to the
// *type* of the parameter in question. If two arguments are
// given, like here, the first is assigned the index into the
// parameter tuple, and the second to the *type* of the
// corresponding parameter.
foreach (i, T; Params) {
// This quirky behaviour of foreach on parameter tuples
// means T is *not* the parameter itself, merely the
// *type* of the parameter. To get the actual
// parameter's description, we need a 1-element slice
// of the parameter tuple. Thankfully, i has been
// conveniently assigned the index of the current
// parameter, so we can use that for the slicing:
alias Param = Params[i .. i+1];
// As I said, T is merely the type of the parameter,
// not the parameter itself.
alias ParamType = T;
// The 1-element slice Param behaves as if it's an
// independent entity with a name, so we can use
// __traits(identifer, ...) to get at the parameter
// name.
string ParamName = __traits(identifier, Param);
writeln(type=, T.stringof, name=, ParamName);
// Not shown here is the default parameter value.
// Looking at std.traits.ParameterDefaultValueTuple,
// there appears to be no direct way to get at this.
// The only way seems to be to declare a function with
// Param as its single argument, and then call that
// function with no arguments so that the function can
// return the default value of the parameter. This is
// what Phobos actually does to build the default
// parameter tuple.
// Also not shown here is how to get at the
// storage class of the parameter; I didn't look
// into this, but your best reference would be
// std.traits.ParameterStorageClass' code.
}
}
}
-snip-
Given the non-intuitive quirkiness of the whole thing, you're probably
far better off using Phobos' std.traits.Parameter...Tuple templates to
get at the various bits of function parameters.
Or just parse the string returned by the .stringof value, as you
suggested.
T
--
Never wrestle a pig. You both get covered in mud, and the pig likes it.
Re: Safety/purity and assert/enforce error messages
On Thursday, September 12, 2013 22:17:30 Joseph Rushton Wakeling wrote: On 12/09/13 22:07, Jonathan M Davis wrote: format can't be nothrow, because it throws when you screw up the format specifiers. You have to wrap it in a try-catch block and assert(0) in the catch block if you want to put it in a nothrow function. std.datetime does this in at least a few places. The annoyance of this is that it means that any otherwise @safe pure nothrow function that has an assert() in it can be blocked from being nothrow if it needs a formatted string for the assert failure message. Note that this restriction still applies even if the code is being compiled with -release and the assertion therefore stripped out. Or am I missing a trick as to how to deal with assert() and enforce() ... ? :-) You can put the try-catch in a version(assert) block to get around that problem, but it is true that it's not exactly ideal. However, there really isn't any way around that with a function that takes a format string unless you want it to ignore bad arguments. You'd need a function like text which just appends all of its arguments together in order to get around that problem. - Jonathan M Davis
Re: Adding libraries to an executable
On 2013-09-12 18:25, Anton Alexeev wrote: [Environment] DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import -L-L/usr/lib/i386-linux-gnu -L-L/usr/lib/x86_64-linux-gnu -L--no-warn-search-mismatch -L--export-dynamic I don't see way explicitly linking with Phobos would make a difference, since the compiler will link with Phobos implicitly. That assumes you have libphobos2.a in /usr/lib/i386-linux-gnu or /usr/lib/x86_64-linux-gnu. -- /Jacob Carlborg
Re: Greedy memory handling
On Thu, Sep 12, 2013 at 07:50:25PM +0400, Dmitry Olshansky wrote:
12-Sep-2013 17:51, H. S. Teoh пишет:
[...]
struct WeakPointer(T) {
enum size_t mask = 0xdeadbeef;
union Impl {
T* ptr;
size_t uintVal;
}
Impl impl;
void set(T* ptr) @system {
impl.ptr = ptr;
impl.uintVal ^= mask;
}
T* get() @system {
Impl i = impl;
i.uintVal ^= mask;
return i.ptr;
}
}
WeakPointer!Buffer bufferRef;
void doWork(Args...) {
T* buffer;
if (bufferRef.get() is null) {
// Buffer hasn't been allocated yet
buffer = allocateNewBuffer();
bufferRef.set(buffer);
} else {
void *p;
core.memory.GC.getAttr(p);
This line above is not 100% good idea .. at least with deadbeaf as
mask.
If we do know what OS you compile for we may just flip the say upper
bit and get a pointer into kernel space (and surely that isn't in GC
pool). Even then your last paragraph pretty much destroys it.
Well, that was just an example value. :) If we know which OS it is and
how it assigns VM addresses, then we can adjust the mask appropriately.
But yeah, calling GC.getAttr is unreliable since you can't tell whether
the block is what you had before, or somebody else's new data.
[...]
Better option is to have finalizer hooked up to set some flag. Then
_after_ restoring the pointer we consult that flag variable.
Good idea. The problem is, how to set a finalizer on a memory block that
can change in size? The OP's original situation was that the buffer can
be extended while in use, but I don't know of any D type that can
associate a dtor with a ubyte[] array (note that the GC collecting the
wrapper struct/class around the ubyte[] is not the same as collecting
the actual memory block storing the ubyte[] -- the former can happen
without the latter).
T
--
People tell me that I'm skeptical, but I don't believe it.
Re: Greedy memory handling
On 2013-09-12 15:51, H. S. Teoh wrote: The problem is, he wants to reuse the buffer next time if the GC hasn't collected it yet. I was thinking he could reuse the stack/static buffer. Basically using two buffers, one static and one dynamic. -- /Jacob Carlborg
Re: Safety/purity and assert/enforce error messages
On 12/09/13 22:07, Jonathan M Davis wrote: format can't be nothrow, because it throws when you screw up the format specifiers. You have to wrap it in a try-catch block and assert(0) in the catch block if you want to put it in a nothrow function. std.datetime does this in at least a few places. The annoyance of this is that it means that any otherwise @safe pure nothrow function that has an assert() in it can be blocked from being nothrow if it needs a formatted string for the assert failure message. Note that this restriction still applies even if the code is being compiled with -release and the assertion therefore stripped out. Or am I missing a trick as to how to deal with assert() and enforce() ... ? :-)
Re: Greedy memory handling
On Thu, Sep 12, 2013 at 11:13:30PM +0400, Dmitry Olshansky wrote:
12-Sep-2013 20:51, H. S. Teoh пишет:
On Thu, Sep 12, 2013 at 07:50:25PM +0400, Dmitry Olshansky wrote:
[...]
Better option is to have finalizer hooked up to set some flag. Then
_after_ restoring the pointer we consult that flag variable.
Good idea. The problem is, how to set a finalizer on a memory block
that can change in size? The OP's original situation was that the
buffer can be extended while in use, but I don't know of any D type
that can associate a dtor with a ubyte[] array (note that the GC
collecting the wrapper struct/class around the ubyte[] is not the
same as collecting the actual memory block storing the ubyte[] -- the
former can happen without the latter).
Double indirection? Allocate a class that has finalizer, hold that
via weak-ref. The wrapper in turn contains a pointer to the buffer.
The interesting point then is that one may allocate said buffer via
C's realloc.
Then once helper struct is collected the finalizer is called and
this is where we call free to cleanup C's heap.
I'm thinking this actually is going to work.
[...]
Interesting idea, use C's malloc/realloc to hold the actual buffer. Only
possible catch is, will that cause the GC to collect when it runs out of
memory (which is the whole point of the OP's question)? I.e., does it
make a difference in GC behaviour to allocate, say, 10MB from the GC vs.
allocating 10MB from malloc/realloc?
Assuming we have that settled, something like this should work:
bool isValid;
final class BufWrapper {
void* ptrToMallocedBuf;
this(void* ptr) {
// We need this, 'cos otherwise we don't know if
// our weak ref to BufWrapper is still valid!
isValid = true;
ptrToMallocedBuf = ptr;
}
~this() {
// If we're being collected, free the real
// buffer too.
free(ptrToMallocedBuf);
isValid = false;
}
}
// WeakPointer masks the pointer to BufWrapper in some suitable
// way so that the GC will collect it when needed.
WeakPointer!BufWrapper wrappedBufRef;
void doWork(...) {
void* buf;
if (!isValid) {
buf = realloc(null, bufSize);
wrappedBufRef.set(buf);
} else {
buf = wrappedBufRef.get();
}
// use buf here.
}
T
--
Public parking: euphemism for paid parking. -- Flora
Linking Trouble (Linux)
I just upgraded my Linux distro (openSuse) and now when trying to compile a project with dub I am getting linking errors. Compilation goes OK until the linking stage, when I get the following: Linking... dmd -of/home/craig/cloud/vibe-tiles/vibe /home/craig/cloud/vibe-tiles/temp.o -L-levent_pthreads -L-levent -L-lssl -L-lcrypto -g /usr/lib64/gcc/x86_64-suse-linux/4.7/../../../../x86_64-suse-linux/bin/ld: warning: libdl.so.2, needed by /usr/lib64/libcrypto.so, not found (try using -rpath or -rpath-link) /usr/lib64/gcc/x86_64-suse-linux/4.7/../../../../x86_64-suse-linux/bin/ld: warning: libz.so.1, needed by /usr/lib64/libcrypto.so, not found (try using -rpath or -rpath-link) /usr/lib64/libcrypto.so: undefined reference to `dlopen@GLIBC_2.2.5' /usr/lib64/libcrypto.so: undefined reference to `dlerror@GLIBC_2.2.5' /usr/lib64/libcrypto.so: undefined reference to `dlclose@GLIBC_2.2.5' /usr/lib64/libcrypto.so: undefined reference to `dlsym@GLIBC_2.2.5' /usr/lib64/libcrypto.so: undefined reference to `dladdr@GLIBC_2.2.5' Strangely, libdl.so.2 and libz.so.1 are both there and libcrypto is linked to them. $ ldd /usr/lib64/libcrypto.so linux-vdso.so.1 (0x7fff7d52) libdl.so.2 = /lib64/libdl.so.2 (0x7f3b2ef98000) libz.so.1 = /lib64/libz.so.1 (0x7f3b2ed8) libc.so.6 = /lib64/libc.so.6 (0x7f3b2e9d) /lib64/ld-linux-x86-64.so.2 (0x7f3b2f5a8000) I've surfed around quite a bit and can't seem to find anything. I solved a similar problem in the past by reinstalling glibc, but that hasn't helped in this case. Any hints on where to look would be appreciated. Craig
Re: Safety/purity and assert/enforce error messages
On 12/09/13 20:49, H. S. Teoh wrote: In phobos git HEAD, std.format has been made pure @safe nothrow (and CTFE-able), so you should be able to write your assert as: assert(condition, format(x = %s, blahblah, x)); Don't think it will work on 2.063.2, though, since it was a relatively recent change. I suspect this will only work starting from 2.064. Ahh, good to know, thanks. I guess I'll withhold tweaking such things until after the next release ... :-)
Re: Greedy memory handling
13-Sep-2013 00:11, H. S. Teoh пишет:
On Thu, Sep 12, 2013 at 11:13:30PM +0400, Dmitry Olshansky wrote:
12-Sep-2013 20:51, H. S. Teoh пишет:
On Thu, Sep 12, 2013 at 07:50:25PM +0400, Dmitry Olshansky wrote:
[...]
Better option is to have finalizer hooked up to set some flag. Then
_after_ restoring the pointer we consult that flag variable.
Good idea. The problem is, how to set a finalizer on a memory block
that can change in size? The OP's original situation was that the
buffer can be extended while in use, but I don't know of any D type
that can associate a dtor with a ubyte[] array (note that the GC
collecting the wrapper struct/class around the ubyte[] is not the
same as collecting the actual memory block storing the ubyte[] -- the
former can happen without the latter).
Double indirection? Allocate a class that has finalizer, hold that
via weak-ref. The wrapper in turn contains a pointer to the buffer.
The interesting point then is that one may allocate said buffer via
C's realloc.
Then once helper struct is collected the finalizer is called and
this is where we call free to cleanup C's heap.
I'm thinking this actually is going to work.
[...]
Interesting idea, use C's malloc/realloc to hold the actual buffer. Only
possible catch is, will that cause the GC to collect when it runs out of
memory (which is the whole point of the OP's question)? I.e., does it
make a difference in GC behaviour to allocate, say, 10MB from the GC vs.
allocating 10MB from malloc/realloc?
The only problem I can foresee is that when it runs the collection
(*and* being tight on RAM) the C heap will not return said chunk back to
OS. Then GC won't pick up that memory, and we'd get out of ram.
I would safely assume however that for big buffers a mmap/munmap is
called (or its analogue) and hence memory is returned back to OS. That's
what all allocators do for huge chunks by anyway.
Otherwise we are still in a good shape, the memory will eventually be
freed, yet we get to reuse it quite cheaply in a tight loop. I don't
expect collections to run in these all that often ;)
Assuming we have that settled, something like this should work:
bool isValid;
final class BufWrapper {
void* ptrToMallocedBuf;
this(void* ptr) {
// We need this, 'cos otherwise we don't know if
// our weak ref to BufWrapper is still valid!
isValid = true;
ptrToMallocedBuf = ptr;
}
~this() {
// If we're being collected, free the real
// buffer too.
free(ptrToMallocedBuf);
isValid = false;
}
}
// WeakPointer masks the pointer to BufWrapper in some suitable
// way so that the GC will collect it when needed.
WeakPointer!BufWrapper wrappedBufRef;
void doWork(...) {
void* buf;
Careful here - you really have first to get a pointer ... THEN check if
it's valid.
if (!isValid) {
buf = realloc(null, bufSize);
wrappedBufRef.set(buf);
} else {
//otherwise at this point GC.collect runs and presto, memory is freed
//too bad such a thing will never show up in unittests
buf = wrappedBufRef.get();
}
// use buf here.
}
Checking the flag should be somehow part of weak ref job.
I'd rather make it less error prone:
void* buf;
//unmask pointer, do the flag check - false means was freed
if(!weakRef.readTo(buf)){
//create set new buf
buf = realloc(...);
}
... //use buf
weakRef.set(buf);
I think I'd code it up if nobody beats me to it as I need the same exact
pattern for std.regex anyway.
--
Dmitry Olshansky
Re: What does __parameters return?
On Thursday, 12 September 2013 at 19:41:26 UTC, H. S. Teoh wrote: On Thu, Sep 12, 2013 at 08:44:29PM +0200, simendsjo wrote: (...) .. but what type is it? Error: argument (int i, char c, string s, bool b = false) to typeof is not an expression How am I supposed to get the parameter names using this? (...) You're not going to like the answer to this one... :) (...) Or just parse the string returned by the .stringof value, as you suggested. Thanks. I just ended up using both is(__parameters and is(function to fetch the parameter names. Sometimes it seems things are added to D without a very thorough design phase, but then again, it starts with __, so I guess I'm on my own :)
Re: override and package
On Thursday, 12 September 2013 at 16:16:14 UTC, Maxim Fomin wrote: On Thursday, 12 September 2013 at 11:29:22 UTC, Jacob Carlborg wrote: On 2013-09-12 11:28, Namespace wrote: But if I try to write 'override' before [1], I get this error message: Error: function T4._apply cannot override a non-virtual function This seems inconsistent. I really overwrite the method, and then I put it in a package label. I think the error message is pretty clear. You cannot override a function that isn't virtual. Private and package methods are not virtual. Actually error message is misleading - it complains as T1._apply would be non-virtual function which is wrong, it is one. The problem is not in case of overriding non-virtual function but in attempt to override virtual function (private) by non-virtual one (package). I think error mesage should be something like: Error: function T4._apply is non-virtual function and cannot ovverride virtual function T1._apply Thank you, you understand my problem. ;) I agree, the message should be changed.
Re: Safety/purity and assert/enforce error messages
On Thu, Sep 12, 2013 at 08:13:37PM +0200, Joseph Rushton Wakeling wrote: Hi all, Suppose that I want to insert some variable values into an assert error message. The easy way to do this is something like: assert(someCondition, text(x = , x, , cannot blah blah blah)); However, the use of text() prevents me from applying @safe and other such attributes to the function containing this assert, even though text() will only be called in the event of an assertion failure. Is there any reliable way to include variable values in the error message that doesn't interfere with @safe, pure, etc.? Using to!string has a similar problem. [...] In phobos git HEAD, std.format has been made pure @safe nothrow (and CTFE-able), so you should be able to write your assert as: assert(condition, format(x = %s, blahblah, x)); Don't think it will work on 2.063.2, though, since it was a relatively recent change. I suspect this will only work starting from 2.064. T -- MASM = Mana Ada Sistem, Man!
Re: What does __parameters return?
On 9/12/13, simendsjo simend...@gmail.com wrote: Thanks. I just ended up using both is(__parameters and is(function to fetch the parameter names. Sometimes it seems things are added to D without a very thorough design phase, but then again, it starts with __, so I guess I'm on my own :) Have you tried using ParameterIdentifierTuple from std.traits?
Re: Linking Trouble (Linux)
On Thursday, 12 September 2013 at 20:26:29 UTC, Craig Dillabaugh
wrote:
On Thursday, 12 September 2013 at 19:46:21 UTC, Craig Dillabaugh
wrote:
I just upgraded my Linux distro (openSuse) and now when trying
to
compile a project with dub I am getting linking errors.
Compilation goes OK until the linking stage, when I get the
following:
Linking...
dmd -of/home/craig/cloud/vibe-tiles/vibe
/home/craig/cloud/vibe-tiles/temp.o -L-levent_pthreads
-L-levent
-L-lssl -L-lcrypto -g
clip
Strangely, libdl.so.2 and libz.so.1 are both there and
libcrypto
is linked to them.
clip
Craig
OK, so I solved my compilation problem by passing the following
as the linker command (added -L-ldl and -L-lz):
dmd -of/home/craig/cloud/vibe-tiles/vibe
/home/craig/cloud/vibe-tiles/temp.o -L-levent_pthreads -L-levent
-L-lssl -L-ldl -L-lz -L-lcrypto -g
How the GCC linker decides which libraries I need to explicitly
specify is indeed a mystery to me.
Craig
For anyone who runs into a similar problem using DUB the
following package.json files solves the problem:
{
name: vibe,
description: My first vibe project.,
homepage: http://example.org;,
copyright: Me,
authors: [
Craig Dillabaugh
],
dependencies: {
vibe-d: ~master
},
libs: [dl, z]
}
Re: What does __parameters return?
On Thu, Sep 12, 2013 at 10:40:06PM +0200, simendsjo wrote: On Thursday, 12 September 2013 at 19:41:26 UTC, H. S. Teoh wrote: On Thu, Sep 12, 2013 at 08:44:29PM +0200, simendsjo wrote: (...) .. but what type is it? Error: argument (int i, char c, string s, bool b = false) to typeof is not an expression How am I supposed to get the parameter names using this? (...) You're not going to like the answer to this one... :) (...) Or just parse the string returned by the .stringof value, as you suggested. Thanks. I just ended up using both is(__parameters and is(function to fetch the parameter names. Sometimes it seems things are added to D without a very thorough design phase, but then again, it starts with __, so I guess I'm on my own :) I think Walter's official statement on the matter is that __traits is not intended to be used by user code, but to allow Phobos to access compiler internals so that it can provide more user-friendly wrappers around the raw functionality. T -- Nobody is perfect. I am Nobody. -- pepoluan, GKC forum
Re: Safety/purity and assert/enforce error messages
On 12/09/13 22:23, Jonathan M Davis wrote: You can put the try-catch in a version(assert) block to get around that problem, but it is true that it's not exactly ideal. However, there really isn't any way around that with a function that takes a format string unless you want it to ignore bad arguments. You'd need a function like text which just appends all of its arguments together in order to get around that problem. Well, I have been using text() to generate my assert error messages, but the trouble is that (at least in 2.063) it's not @safe, pure or nothrow ... :-(
Re: Safety/purity and assert/enforce error messages
On Thu, Sep 12, 2013 at 11:37:10PM +0200, Joseph Rushton Wakeling wrote: On 12/09/13 22:23, Jonathan M Davis wrote: You can put the try-catch in a version(assert) block to get around that problem, but it is true that it's not exactly ideal. However, there really isn't any way around that with a function that takes a format string unless you want it to ignore bad arguments. You'd need a function like text which just appends all of its arguments together in order to get around that problem. Well, I have been using text() to generate my assert error messages, but the trouble is that (at least in 2.063) it's not @safe, pure or nothrow ... :-( Even in git HEAD, text() may still throw. So ultimately, you still need assumeWontThrow: https://github.com/D-Programming-Language/phobos/pull/1571 :) T -- Some days you win; most days you lose.
Re: isRandomAccessRange!(const(size_t[])) is false ... ?
On 09/12/2013 02:53 PM, Joseph Rushton Wakeling wrote:
import std.range;
void main()
{
assert(isRandomAccessRange!(const(size_t[])));
}
results in an assertion error. This is a bug, no ... ?
That one is fine. You probably want a slice of const(size_t):
import std.range;
void main()
{
assert(!isRandomAccessRange!(const(size_t[])));
assert( isRandomAccessRange!(const(size_t)[])); // --
}
Ali
isRandomAccessRange!(const(size_t[])) is false ... ?
import std.range;
void main()
{
assert(isRandomAccessRange!(const(size_t[])));
}
results in an assertion error. This is a bug, no ... ?
Re: Safety/purity and assert/enforce error messages
On Thursday, 12 September 2013 at 20:07:40 UTC, Jonathan M Davis
wrote:
On Thursday, September 12, 2013 12:42:34 H. S. Teoh wrote:
On Thu, Sep 12, 2013 at 09:12:18PM +0200, bearophile wrote:
H. S. Teoh:
In phobos git HEAD, std.format has been made pure @safe
nothrow (and
CTFE-able), so you should be able to write your assert as:
assert(condition, format(x = %s, blahblah, x));
With the latest DMD from updated GIT head:
import std.string: format;
void main() pure nothrow {
string x = hello;
bool condition = true;
assert(condition, format(x = %s, blahblah, x));
}
It gives:
test.d(5): Error: 'std.string.format!(char, string).format'
is not nothrow
test.d(2): Error: function 'D main' is nothrow yet may throw
[...]
Oops. Apparently I wrote nothrow but forgot to test it. I did
test pure
and @safe, though, so at least those two should work.
format can't be nothrow, because it throws when you screw up
the format
specifiers. You have to wrap it in a try-catch block and
assert(0) in the catch
block if you want to put it in a nothrow function. std.datetime
does this in
at least a few places.
- Jonathan M Davis
Why don't we have a format function that is statically checked?
Re: isRandomAccessRange!(const(size_t[])) is false ... ?
On Thursday, September 12, 2013 23:53:15 Joseph Rushton Wakeling wrote:
import std.range;
void main()
{
assert(isRandomAccessRange!(const(size_t[])));
}
results in an assertion error. This is a bug, no ... ?
Nope. It's correct. Because it's const, you can't call popFront on it or any
of the other mutating range operations. So, it's not a valid range.
- Jonathan M Davis
Re: isRandomAccessRange!(const(size_t[])) is false ... ?
On Thu, Sep 12, 2013 at 03:12:48PM -0700, Ali Çehreli wrote:
On 09/12/2013 02:53 PM, Joseph Rushton Wakeling wrote:
import std.range;
void main()
{
assert(isRandomAccessRange!(const(size_t[])));
}
results in an assertion error. This is a bug, no ... ?
That one is fine. You probably want a slice of const(size_t):
import std.range;
void main()
{
assert(!isRandomAccessRange!(const(size_t[])));
assert( isRandomAccessRange!(const(size_t)[])); // --
}
[...]
Explanation:
const(size_t)[] means the elements of the array can't be modified, but
the array itself can (necessary to implement popFront, popBack, etc.).
const(size_t[]) means the array itself can't be modified, so popFront,
popBack are invalid. So it cannot be a range.
T
--
Food and laptops don't mix.
Re: Safety/purity and assert/enforce error messages
On Thu, Sep 12, 2013 at 11:45:07PM +0200, Meta wrote:
On Thursday, 12 September 2013 at 20:07:40 UTC, Jonathan M Davis
wrote:
[...]
format can't be nothrow, because it throws when you screw up the
format specifiers. You have to wrap it in a try-catch block and
assert(0) in the catch block if you want to put it in a nothrow
function. std.datetime does this in at least a few places.
- Jonathan M Davis
Why don't we have a format function that is statically checked?
Currently, this is not possible, because the format string may be a
runtime-computed value. For example:
void main(string[] args) {
// fmtString is only known at runtime.
auto fmtString = File(fmtstrings.cfg).byLine().front.idup;
// This will throw if command-line arguments don't match
// format string.
writefln(fmtString, args[1..$]);
}
It *is* true, however, that the vast majority of format usage uses a
static format string. In those cases, it *would* be nice to have a
nothrow variant of format(). Arguably, we could have this variant of
format:
string staticFormat(string fmt, T...)(T args) { ... }
which takes a compile-time format string. Then it can be statically
verified to be pure, @safe, nothrow, etc..
This is actually more tricky than it sounds, though. Specifiers like
%s are parsed into Format!char objects, which may be passed to a
user-defined type's toString method, so the nothrow-ness of
staticFormat() may change depending on whether the user-defined toString
throws.
T
--
Старый друг лучше новых двух.
Re: isRandomAccessRange!(const(size_t[])) is false ... ?
On 9/12/13, Joseph Rushton Wakeling joseph.wakel...@webdrake.net wrote:
import std.range;
void main()
{
assert(isRandomAccessRange!(const(size_t[])));
}
results in an assertion error. This is a bug, no ... ?
I think ranges have to be non-const. I mean how can you iterate over
one if e.g. popFront can't manipulate the inner state?
Maybe you meant:
const(size_t)[]
P.S. you can use a 'static assert' in your code.
Re: isRandomAccessRange!(const(size_t[])) is false ... ?
On 13/09/13 00:15, Jonathan M Davis wrote:
Nope. It's correct. Because it's const, you can't call popFront on it or any
of the other mutating range operations. So, it's not a valid range.
D'oh! :-P
I gave the assert as an example, but this one snuck up on me rather unexpectedly
-- I'd got a class method that was something like this:
auto foo(in size_t n) @safe const nothrow pure
{
return buf[n]; // where buf is a size_t[][]
}
... and it turned out that foo was trying to return a const(size_t[]) rather
than a size_t[].
Re: isRandomAccessRange!(const(size_t[])) is false ... ?
On Friday, September 13, 2013 00:46:20 Joseph Rushton Wakeling wrote:
On 13/09/13 00:15, Jonathan M Davis wrote:
Nope. It's correct. Because it's const, you can't call popFront on it or
any of the other mutating range operations. So, it's not a valid range.
D'oh! :-P
I gave the assert as an example, but this one snuck up on me rather
unexpectedly -- I'd got a class method that was something like this:
auto foo(in size_t n) @safe const nothrow pure
{
return buf[n]; // where buf is a size_t[][]
}
... and it turned out that foo was trying to return a const(size_t[]) rather
than a size_t[].
At present, const and ranges basically don't mix. Slicing an array results in
a tail-const slice, which allows you to get away with using const arrays and
then slice them when you need to operate on a range which is const, but it's
actually really, really hard to define opSlice in a way that that works for
user-defined types. We really need to fix that, but for now, it generally means
that you don't use const if you're using ranges (which of course is not what
we want, but we're stuck for the moment).
- Jonathan M Davis
Re: Safety/purity and assert/enforce error messages
On Thursday, 12 September 2013 at 22:04:20 UTC, H. S. Teoh wrote:
Currently, this is not possible, because the format string may
be a runtime-computed value. For example:
...
When you don't have a runtime computed format string, though
(which is
generally the case for my code, at least), it would be extremely
useful
for a literal format string to be statically verified and
nothrow. One really nice thing I've seen with the Rust language
is that they have a fmt macro that statically verifies that the
types of its arguments match the format string.
It *is* true, however, that the vast majority of format usage
uses a
static format string. In those cases, it *would* be nice to
have a
nothrow variant of format(). Arguably, we could have this
variant of
format:
string staticFormat(string fmt, T...)(T args) { ... }
which takes a compile-time format string. Then it can be
statically
verified to be pure, @safe, nothrow, etc..
This is actually more tricky than it sounds, though. Specifiers
like
%s are parsed into Format!char objects, which may be passed
to a
user-defined type's toString method, so the nothrow-ness of
staticFormat() may change depending on whether the user-defined
toString
throws.
Is it necessary to allow toString to throw? I can't think of any
reason other than the object being in an invalid state, in which
case maybe it should throw an Error.
