Re: DUnit: Advanced unit testing toolkit.
On Monday, 23 September 2013 at 16:40:56 UTC, jostly wrote:
In specd, the actual code inside the unittest { } sections only
collect results, and the reporting is called from a main()
supplied by compiling with version specrunner set. I haven't
checked to see if your dunit do something similar.
I think more D-way would have been to simply separate tests in
logically separated unittest blocks and run those using static
reflection, catching assert error. That way you will get first
failure in a set and then continue to other sets. Does that make
sense?
Re: DUnit: Advanced unit testing toolkit.
On Sunday, 22 September 2013 at 13:13:29 UTC, linkrope wrote:
Have a look at https://github.com/linkrope/dunit, especially at
the Related Projects.
Until now, my preferred tool for (large-scale) unit testing in D
would be the combination of my dunit framework (of course),
DMocks-revived for mocks, and the 'must' matchers of specd.
I think it's great to see the D unit testing ecosystem growing.
Since it's still relatively small, I think we have a good chance
here to create interoperability between the different frameworks.
As I see it, we have:
1. Running unit tests
This is where D shines with the builting facility for unit tests.
However, it suffers a bit from the fact that, if we use assert,
it will stop on the first assertion failure, and there is (as far
as I've been able to tell) no reliable way to run specific code
before or after all the unit tests. If I'm wrong on that
assumption, please correct me, that would simplify the spec
running for specd.
In specd, the actual code inside the unittest { } sections only
collect results, and the reporting is called from a main()
supplied by compiling with version specrunner set. I haven't
checked to see if your dunit do something similar.
2. Asserting results
Varies from the builtin assert() to xUnit-like assertEquals() to
the more verbose x.must.equal(y) used in specd.
This could easily be standardized by letting all custom asserts
throw an AssertError, though I would prefer to use another
exception that encapsulates the expected and actual result, to
help with bridging to reporting.
3. Reporting results
If we have moved beyond basic assert() and use some kind of unit
test runner, then we have the ability to report a summary of run
tests, and which (and how many) failed.
This is one area where IDE integration would be very nice, and I
would very much prefer it if the different unit test frameworks
agreed on one standard unit test runner interface, so that the
IDE integration problem becomes one of adapting each IDE to one
runner interface, instead of adapting each framework to each IDE.
In my experience from the Java and Scala world, the last point is
the biggest. Users expect to be able to run unit tests and see
the report in whatever standard way their IDE has. In practice
this most often means that various libraries pretend to be JUnit
when it comes to running tests, because JUnit is supported by all
IDEs.
Let's not end up in that situation, but rather work out a common
API to run unit tests, and the D unit test community can be the
envy of every other unit tester. :)
Re: DUnit: Advanced unit testing toolkit.
On Mon, 2013-09-23 at 18:40 +0200, jostly wrote:
[…]
I think it's great to see the D unit testing ecosystem growing.
Since it's still relatively small, I think we have a good chance
here to create interoperability between the different frameworks.
There is also integration and system testing of course, not just unit
testing. The same testing framework can generally be used for all forms.
In the Java sphere, JUnit gave way to TestNG exactly because TestNG
supports all forms of testing not just unit testing. Now though TestNG
is giving way to Spock enabling elements of BDD as well as TDD.
D has some unit testing capability built in, which i good, but it is
also good to have an external testing framework that can do unit,
integration and system testing supporting TDD and BDD.
As I see it, we have:
1. Running unit tests
This is where D shines with the builting facility for unit tests.
However, it suffers a bit from the fact that, if we use assert,
it will stop on the first assertion failure, and there is (as far
as I've been able to tell) no reliable way to run specific code
before or after all the unit tests. If I'm wrong on that
assumption, please correct me, that would simplify the spec
running for specd.
So the built-in is not entirely up to the task of real unit testing?
In specd, the actual code inside the unittest { } sections only
collect results, and the reporting is called from a main()
supplied by compiling with version specrunner set. I haven't
checked to see if your dunit do something similar.
2. Asserting results
Varies from the builtin assert() to xUnit-like assertEquals() to
the more verbose x.must.equal(y) used in specd.
In the Scala variant of the JVM arena, ScalaTest mingles really nicely
all the classic TDD asserts styles, along with Hamcrest matchers, but
also supports the more BDD style test specifications. Spock also does
this. Corollary, D must do this.
NB Go is going through all this just now. The built in unit test
capability is minimalist and for testing the Go implementation. GoCheck
is classic TDD assert style, and there are some candidate BDD styles on
the horizon. Will Go beat D to having the capability that the JVM
languages already enjoy?
Note that Groovy and the py.test Python test framework dispense with the
need for assertEquals and that family of JUnit thingies, in favour of
using the built-in assert and catching the AssertionError exception,
doing detailed stack analysis and provided very detailed information
about the evaluated expression: power asserts. Why should D follow 1990s
thinking when there is 2010s thinking that is much better?
This could easily be standardized by letting all custom asserts
throw an AssertError, though I would prefer to use another
exception that encapsulates the expected and actual result, to
help with bridging to reporting.
See above :-)
3. Reporting results
If we have moved beyond basic assert() and use some kind of unit
test runner, then we have the ability to report a summary of run
tests, and which (and how many) failed.
This is one area where IDE integration would be very nice, and I
would very much prefer it if the different unit test frameworks
agreed on one standard unit test runner interface, so that the
IDE integration problem becomes one of adapting each IDE to one
runner interface, instead of adapting each framework to each IDE.
In my experience from the Java and Scala world, the last point is
the biggest. Users expect to be able to run unit tests and see
the report in whatever standard way their IDE has. In practice
this most often means that various libraries pretend to be JUnit
when it comes to running tests, because JUnit is supported by all
IDEs.
Let's not end up in that situation, but rather work out a common
API to run unit tests, and the D unit test community can be the
envy of every other unit tester. :)
Currently, and very sadly, this generally means writing an XML file
using the JUnit schema. On the other hand if D did this Eclipse, IDEA,
NetBeans, etc. would immediately render excellent data displays.
--
Russel.
=
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
signature.asc
Description: This is a digitally signed message part
Re: DUnit: Advanced unit testing toolkit.
On Monday, 23 September 2013 at 16:40:56 UTC, jostly wrote:
Let's not end up in that situation, but rather work out a
common API to run unit tests, and the D unit test community can
be the envy of every other unit tester. :)
You've raised some nice ideas and got me thinking. However, i do
think we are missing some way of knowing when unit tests start
and stop. I like the built in unittest blocks but it would be
nice to have something like:
beforetests
{
...
}
aftertests
{
...
}
To apply code before and after the unit tests have run. These
could be used to setup and the execute the reporting environment?
I can't think of a way to do this automatically without these
constructs.
Re: DUnit: Advanced unit testing toolkit.
Can you not just change to going through each module running the hosts
manually? Then you can do what you like before and after each.
If your lib had a host that started a thread which received messages about
a unittest registering itself in other user modules you could run the
unittests as you feel like and you could stick with the -unittest dmd arg.
Reason for the thread is you can't control order of module initialization.
On 23 Sep 2013 21:30, Gary Willoughby d...@nomad.so wrote:
On Monday, 23 September 2013 at 16:40:56 UTC, jostly wrote:
Let's not end up in that situation, but rather work out a common API to
run unit tests, and the D unit test community can be the envy of every
other unit tester. :)
You've raised some nice ideas and got me thinking. However, i do think we
are missing some way of knowing when unit tests start and stop. I like the
built in unittest blocks but it would be nice to have something like:
beforetests
{
...
}
aftertests
{
...
}
To apply code before and after the unit tests have run. These could be
used to setup and the execute the reporting environment? I can't think of a
way to do this automatically without these constructs.
Re: DUnit: Advanced unit testing toolkit.
Sorry in my last mail host should be unittest (swype auto correct errors :))
Re: Will Java go native?
On Sunday, 22 September 2013 at 02:04:11 UTC, Paul Jurczak wrote: On Thursday, 19 September 2013 at 09:48:15 UTC, Joakim wrote: On Thursday, 19 September 2013 at 08:26:03 UTC, Russel Winder wrote: Java is no longer under-performant compared to C, C++, Fortran, D, Go, Rust. Check the benchmarks. Interesting. Java people have been saying this for years and it's never been quite true, so I just looked up the benchmark shootout to see if you're right. It shows Java 7 doing pretty well against C++ these days, roughly equivalent speed or at worst half as fast: http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?test=alllang=javadata=u64q [...] It doesn't do that well when single core is selected (always slower, 4x worst case): http://benchmarksgame.alioth.debian.org/u64/benchmark.php?test=alllang=javalang2=gppdata=u64 It means that Java VM/libraries can explore multiple cores better that C++. Which in a day and age that you can hardly buy single core machines, is a big advantage. -- Paulo
Re: DIP45: fixing the dllimport/dllexport issue
On 23.09.2013 07:22, Benjamin Thaut wrote: Am 10.09.2013 05:50, schrieb Martin Nowak: Can we deal with vtable entries pointing to imported functions on Windows? The kind of thing that happens when you inherit across a DLL boundary. I can't answer that and with dmd's current implementation of export its not possible to try this. I would say we implement the DIP and see on the way if this becomes a issue. It is the same as initializing a global variable with a pointer into another DLL, i.e. it needs some init code to set the value.
Re: DIP45: fixing the dllimport/dllexport issue
On 19.09.2013 21:42, Benjamin Thaut wrote: Am 19.09.2013 20:24, schrieb Rainer Schuetze: On 19.09.2013 13:58, Benjamin Thaut wrote: Am 12.09.2013 07:54, schrieb Rainer Schuetze: This sounds interesting. Stripping an existing library isn't even needed because it is normally never created with exports anyway (-lib implies that no exports are created to start with). Stripping object files might be necessary, though, in case they have been built separately with -c. Can't we get around stripping object files by always using -c together with -lib in case the object file is supposed to go into a static library? That could work, though it has it's own set of side effects, like splitting a module into pseudo modules per function which also ruins running unittests built into the library. Building modules with -c and combining these into a library afterwards avoids this. Ah ok, I didn't know that. How much work would it be to implement object file stripping? I'm not confident I can do this. I just checked the OMF and COFF docs: it should be possible to wipe out the export records without having to rewrite the object files, so it's not too involved. Don't know about ELF or mach-o, though.
Re: Will Java go native?
On Friday, 20 September 2013 at 07:19:44 UTC, deadalnix wrote:
On Friday, 20 September 2013 at 05:31:09 UTC, Dmitry Leskov
wrote:
Our AOT compiler does inline virtual calls of non-final
methods that are not overloaded in any subclass known at
compile time. Of course, there is a runtime check and a backup
virtual call, but the check is cheap and code size increase in
negligible.
What we don't do (yet) is profile-guided optimizations, but
that is on our to-do list.
So basically what you do is :
funptr = load from vtbl;
if (funptr == compile_time_known_value) {
compile_time_known_value();
} else {
funptr();
}
Which avoid the indirection if the function is actually called
most of the time. Am I right ?
Almost:
funptr = load from vtbl;
if (funptr == compile_time_known_value) {
// Inlined body of compile_time_known_value() goes here
} else {
funptr();
}
Re: std.allocator needs your help
On Monday, 23 September 2013 at 04:19:34 UTC, Manu wrote:
You've just described C++ verbatim.
And you've previously agreed that C++'s approach totally
failed. Can you
explain how this is different from C++, and how it solves the
issues that
defeated that design?
One possibility I'm keeping in mind is that of defining a
dynamic interface
Once you have static allocator you can go dynamic easily:
interface DynamicAllocator // isAllocator==true
{
// whatever the interface is
}
struct StaticAllocator // isAllocator==true
{
}
struct Container(ALLOCATOR = DynamicAllocator) if
(isAllocator!ALLOCATOR)
{
this(ALLOCATOR a){}
}
class DynamicAllocatorWrapper(T) : DynamicAllocator if (is(T) ==
struct)
{
// wraps any static allocator to be callable using dynamic
interface
}
With that boilerplate you can easily create allocator agnostic
containers:
auto c = Container(new
DynamicAllocatorWrapper!(StaticAllocator)())
and you can create specialized version if you need speed very
much:
auto c = Container!(StaticAllocator)(StaticAllocator());
Re: std.allocator needs your help
On 2013-09-23 01:49, Andrei Alexandrescu wrote:
I am making good progress on the design of std.allocator, and I am
optimistic about the way it turns out. D's introspection capabilities
really shine through, and in places the design does feel really
archetypal - e.g. this is the essence of a freelist allocator. It's a
very good feeling. The overall inspiration comes from Berger's
HeapLayers, but D's introspection takes that pattern to a whole new level.
I agree with Manu here. I thought the whole point was to come up with a
design and API how the allocators are supposed to be used. How they
integrate with user code.
Here's a quick idea:
I can think of at least four different usage patterns how an allocator
can be set. Hopefully this will be backwards compatible as well.
1. Globally - The same allocator is used in the whole application by all
threads
2. Thread local - The allocator is set per thread. Different threads can
used different allocators
3. Function local - This is the most intrusive. Sets the allocator for a
given function call
4. Block local - The allocator is used during a given block
This will require some interaction with the runtime and library and
probably use OOP. We define two module variables in some module in
druntime, say rt.allocator:
module rt.allocator:
shared Allocator globalAllocator;
Allocator allocator;
shared this ()
{
globalAllocator = GCAllocator.allocator;
}
The global allocator is, by default, a GC allocator (the GC we're
currently using). For each thread we set the thread local allocator to
be the same as the global allocator:
allocator = globalAllocator;
By default all code will use new to allocate memory. druntime will
have three functions which the new keyword is lowered to. They could
look something like this:
extern (C) ubyte[] _d_tl_new (size_t size)
{
return _d_new(size, rt.allocator.allocator);
}
extern (C) ubyte[] _d_global_new (size_t size)
{
return _d_new(size, rt.allocator.globalAllocator);
}
extern (C) ubyte[] _d_new (size_t size, Allocator allocator)
{
if (memory = allocator.allocate(size))
return memory;
onOutOfMemoryError();
}
By default new is lowered to a call to _d_tl_new, which will
allocate using the thread local allocator, which is by default the same
as the global allocator, that is, the GC. In this way we maintain the
current method of allocating memory.
When using new shared, it's lowered to a function call to
_d_global_new, which uses the global allocator.
For block local allocator an ideal syntax would be:
allocator (GCAllocator.allocator)
{
// explicitly use the GC allocator within this block.
}
If that's not possibly we can define a function look like this:
useAllocator (alias allocator, alias block) ()
{
auto currentAllocator = core.allocator.allocator;
scope (exit)
core.allocator.allocator = currentAllocator;
block();
}
Which is used, something like this:
useAllocator!(GCAllocator.allocator, {
// explicitly use the GC allocator within this block.
});
Or alternately, using a struct:
struct UseAllocator
{
private Allocator currentAlloctor;
this (Allocator allocator)
{
currentAlloctor = core.allocator.allocator;
}
~this ()
{
rt.allocator.allocator = currentAlloctor;
}
}
UseAllocator useAllocator (Allocator allocator)
{
return UseAllocator(allocator);
}
{
auto _ = useAllocator(GCAllocator.allocator);
// explicitly use the GC allocator within this block.
}
Of curse it's also possible to explicitly set the thread local or global
allocator for a more fine grained control. The above functions are just
for convenience and to make sure the allocator is restored.
Function local allocator would just be a function taking an allocator as
a parameter, example:
void process (Allocator) (int a, Allocator allocator =
core.allocator.allocator)
{
auto _ = useAllocator(allocator);
// use the given allocator throughout the rest of this function scope
}
Some bikeshedding:
struct NullAllocator
{
enum alignment = real.alignof;
enum size_t available = 0;
ubyte[] allocate(size_t s) shared { return null; }
bool expand(ref ubyte[] b, size_t minDelta, size_t maxDelta) shared
{ assert(b is null); return false; }
bool reallocate(ref ubyte[] b, size_t) shared
{ assert(b is null); return false; }
void deallocate(ubyte[] b) shared { assert(b is null); }
void collect() shared { }
void deallocateAll() shared { }
static shared NullAllocator it;
}
I would put the asserts in an in block.
* it is an interesting artifact. Allocators may or may not hold
per-instance state. Those that don't are required to define a global
shared or thread-local singleton called it that will be used for all
calls related to allocation. Of course, it is preferred for maximum
flexibility that it is shared so as to clarify that the allocator is
safe to
Re: std.allocator needs your help
On Sunday, 22 September 2013 at 23:49:56 UTC, Andrei Alexandrescu wrote: ... In general I like it though I do agree with concerns mentioned that without speculation about high-level API and usage examples it is rather hard to evaluate its practical applicability. For example, you have mentioned somewhere in comments that it is planned to embed allocator as part of template site like in C++ and it does not really sound a good idea.
Re: std.serialization: pre-voting review / discussion
On 2013-09-21 15:13, mrd wrote: Is this the right way? There are special formats (Protocol Buffers, for example) for a binary format what can be changed over time without breaking old code. But for normal serialization is not this redundant? Besides, search by name slower compared with other methods (field numbers, for example). Not necessarily. I could implement that by default it will use the field number, if the names doesn't match it could fallback to do a lookup by name. I would like to avoid having a dependency on the orders of the fields. -- /Jacob Carlborg
Re: std.serialization: pre-voting review / discussion
On 2013-09-21 14:48, mrd wrote: What is the purpose of the keys? Fields are looked up by name. This is to avoid a dependency of the order of the fields. I guess I can look up by field order instead and fallback to a name look up if a name don't match. If they really needed, can it be realised as option and disable them? (I see also that they are forces to add a lot of duplicate functions.) Yeah, I guess so. I guess if it succeeded binary format can be made very compact (and possibly faster) as Protocol Buffers. I'm working on a binary archive as well. It ignores the name and look up by field order instead. It assume that the archived data and the class/struct has the same field order. -- /Jacob Carlborg
Re: std.allocator needs your help
On Monday, 23 September 2013 at 07:31:46 UTC, Jacob Carlborg wrote: On 2013-09-23 01:49, Andrei Alexandrescu wrote: I am making good progress on the design of std.allocator, and I am optimistic about the way it turns out. D's introspection capabilities really shine through, and in places the design does feel really archetypal - e.g. this is the essence of a freelist allocator. It's a very good feeling. The overall inspiration comes from Berger's HeapLayers, but D's introspection takes that pattern to a whole new level. I agree with Manu here. I thought the whole point was to come up with a design and API how the allocators are supposed to be used. How they integrate with user code. Here's a quick idea: I can think of at least four different usage patterns how an allocator can be set. Hopefully this will be backwards compatible as well. 1. Globally - The same allocator is used in the whole application by all threads 2. Thread local - The allocator is set per thread. Different threads can used different allocators 3. Function local - This is the most intrusive. Sets the allocator for a given function call 4. Block local - The allocator is used during a given block 5. Class local - The allocator is used for specific types (e.g. ASTNode in a compiler) 6. Class-hierarchy - The allocator is used for a specific type hierarchy (e.g. ASTNode might have sub classes Statement,BinOp,etc) 7. Container local - The C++ way which binds allocation to a wrapping container 8. Task local - like Thread local but for std.parallelism.Task That's it for now. This is quite a long list, which means it is probably exhaustive. There should be a generic approach which encompasses at least those cases.
Re: std.allocator needs your help
On 23 September 2013 15:37, Walter Bright newshou...@digitalmars.comwrote: On 9/22/2013 9:19 PM, Manu wrote: Following this train of thought, I can imagine a really nice end goal would be that the existing GC is effectively plugged in as a library, and people can easily substitute it for their own GC if they want/need to. It already is, and has been from the beginning. Rainer, for example, uses a different GC for VisualD. dmd knows naught about the GC. Sure, true. But little things like separate it into its own lib, so you can easily link a different one, or not link one at all. Also, what about language support? How much language support is there currently? I don't think I could currently write the ref-counting GC that I'd like without extensive language support. inc/dec ref calls would need to be inserted all over the place by he compiler, and optimiser would need to know how to remove redundant inc/dec sequences.
Re: std.allocator needs your help
On 9/23/13, Benjamin Thaut c...@benjamin-thaut.de wrote: Why ubyte[] and not void[]? Probably to make the GC avoid scanning into the array.
possible mixin template bug?
This compiles and runs but I think it is a bug.
---
import std.stdio;
mixin template TestMixin(T) {
int testMixin;
}
struct Test {
int testMixin;
mixin TestMixin!Test;
}
void main() {
Test t;
t.testMixin = 10;
writefln(%s, t);
}
---
$ ./hack
Test(10, 0)
I can also change the struct to be:
---
struct Test {
string testMixin;
mixin TestMixin!Test;
}
// and the assignment in main() to
t.testMixin = some string;
---
$ ./hack
Test(some string, 0)
so whether this is a bug or not, I think it is bad. Any thoughts?
G.
Re: [OT] Which IDE / Editor do you use?
On 21/09/2013 16:07, Manu wrote: I'm also not 'average-joe-numskull', at least I don't like to think I am, but that doesn't mean I want to know how a car is built, and then in turn how each individual part was built, and how to fix it, before I can have confidence it will get me to Sydney in one piece. I don't actually really care about how linux works, or any of the little bits and pieces that form it's awkward foundation... and I shouldn't need to in order to like the premise of an open system, and want to use it on that merit alone. I don't actually enjoy OPERATING a computer, I enjoy the creative process of working, and getting work done. Solving interesting problems. If the computer gets in my way, it has failed me at some level. That might sound strange coming from a software engineer, but I guess that's how I see it. I just don't have the patience to mess with my OS anymore. My feelings exactly. I learned about Linux and studied it when I was in high-school (Windows 98/Me era), and I was quite excited about it. Windows was more shite those days, and I knew Linux was not for the average user, but I thought that once I learned it well enough (shell, network, configuring partitions, automounting, the X server, etc.), it would be worthwhile to use. That wasn't the case unfortunately. There was always new stuff that would come up that you would need to learn how to configure, or need to thinker, or there would be shortcomings in application functionality. After a certain point it was just annoying. It might be fun for people who get kicks out of working the innards of a system and being closer to how things work, but on my computer I wanted to either have my leisure time, or get real work done. And spending time configuring stuff (that in Windows just worked out of the box) is not a productive use of one's time in any way, shape or form. True, this was like 10 years ago and Linux distros got better, but so has Windows, and nowdays there is little motivation now for me to try a different OS/desktop-environment. -- Bruno Medeiros - Software Engineer
Re: [OT] Which IDE / Editor do you use?
On Monday, 23 September 2013 at 11:52:28 UTC, Bruno Medeiros wrote: On 21/09/2013 16:07, Manu wrote: ... My feelings exactly. I learned about Linux and studied it when I was in high-school (Windows 98/Me era), and I was quite excited about it. Windows was more shite those days, and I knew Linux was not for the average user, but I thought that once I learned it well enough (shell, network, configuring partitions, automounting, the X server, etc.), it would be worthwhile to use. That wasn't the case unfortunately. There was always new stuff that would come up that you would need to learn how to configure, or need to thinker, or there would be shortcomings in application functionality. After a certain point it was just annoying. It might be fun for people who get kicks out of working the innards of a system and being closer to how things work, but on my computer I wanted to either have my leisure time, or get real work done. And spending time configuring stuff (that in Windows just worked out of the box) is not a productive use of one's time in any way, shape or form. True, this was like 10 years ago and Linux distros got better, but so has Windows, and nowdays there is little motivation now for me to try a different OS/desktop-environment. Ironically, this is exactly the reason I have never succeeded in using the Windows for daily work. Amount of manual configuration and subverting the defaults needed to make it actually usable for my programming flow is outstanding. In the same time on my Linux distro it is mostly `pacman -Sy gnome gnome-extra xorg-server nvidia dlang vim git` and I am ready to work on a fresh install.
Re: possible mixin template bug?
The symbols that are introduced by mixin template never override/conflict
with the formally defined symbols.
It's a designed behavior.
Kenji Hara
2013/9/23 growler growler...@gmail.com
This compiles and runs but I think it is a bug.
---
import std.stdio;
mixin template TestMixin(T) {
int testMixin;
}
struct Test {
int testMixin;
mixin TestMixin!Test;
}
void main() {
Test t;
t.testMixin = 10;
writefln(%s, t);
}
---
$ ./hack
Test(10, 0)
I can also change the struct to be:
---
struct Test {
string testMixin;
mixin TestMixin!Test;
}
// and the assignment in main() to
t.testMixin = some string;
---
$ ./hack
Test(some string, 0)
so whether this is a bug or not, I think it is bad. Any thoughts?
G.
Re: Bartosz Milewski seems to like D more than C++ now :)
On Friday, 20 September 2013 at 15:23:20 UTC, Paulo Pinto wrote: Am 20.09.2013 16:24, schrieb renoX: That said, he made the same mistake as Haskell's authors: currying is a *mathematical detail* which shouldn't obscure function type: 'f: a-b-c' is less readable than 'f: a,b-c'. renoX That is standard in all languages of the ML family, not only Haskell. I know, but 'standard' doesn't mean that it is a good idea.. renoX
Re: Bartosz Milewski seems to like D more than C++ now :)
On Monday, 23 September 2013 at 12:08:48 UTC, renoX wrote: On Friday, 20 September 2013 at 15:23:20 UTC, Paulo Pinto wrote: Am 20.09.2013 16:24, schrieb renoX: That said, he made the same mistake as Haskell's authors: currying is a *mathematical detail* which shouldn't obscure function type: 'f: a-b-c' is less readable than 'f: a,b-c'. renoX That is standard in all languages of the ML family, not only Haskell. I know, but 'standard' doesn't mean that it is a good idea.. renoX I guess it depends on the reader, I always found it quite natural. But then again, I enjoy ML languages since I learned Caml Light. -- Paulo
Re: std.allocator needs your help
On 2013-09-23 11:31, qznc wrote: 5. Class local - The allocator is used for specific types (e.g. ASTNode in a compiler) 6. Class-hierarchy - The allocator is used for a specific type hierarchy (e.g. ASTNode might have sub classes Statement,BinOp,etc) 7. Container local - The C++ way which binds allocation to a wrapping container 8. Task local - like Thread local but for std.parallelism.Task That's it for now. This is quite a long list, which means it is probably exhaustive. There should be a generic approach which encompasses at least those cases. That's a good addition to the list. -- /Jacob Carlborg
Re: std.allocator needs your help
On 9/22/13 9:03 PM, Manu wrote: On 23 September 2013 12:28, Andrei Alexandrescu seewebsiteforem...@erdani.org mailto:seewebsiteforem...@erdani.org wrote: My design makes it very easy to experiment by allowing one to define complex allocators out of a few simple building blocks. It is not a general-purpose allocator, but it allows one to define any number of such. Oh okay, so this isn't really intended as a system then, so much a suggested API? For some definition of system and API, yes :o). That makes almost all my questions redundant. I'm interested in the system, not the API of a single allocator (although your API looks fine to me). I already have allocators I use in my own code. Naturally, they don't inter-operate with anything, and that's what I thought std.allocator was meant to address. Great. Do you have a couple of nontrivial allocators (heap, buddy system etc) that could be adapted to the described API? The proposed design makes it easy to create allocator objects. How they are used and combined is left to the application. Is that the intended limit of std.allocator's responsibility, or will patterns come later? Some higher level design will come later. I'm not sure whether or not you'll find it satisfying, for reasons I'll expand on below. Leaving the usage up to the application means we've gained nothing. I already have more than enough allocators which I use throughout my code. The problem is that they don't inter-operate, and certainly not with foreign code/libraries. This is what I hoped std.allocator would address. Again, if you already have many allocators, please let me know if you can share some. std.allocator will prescribe a standard for defining allocators, with which the rest of std will work, same as std.range prescribes a standard for defining ranges, with which std.algorithm, std.format, and other modules work. Clearly one could come back with but I already have my own ranges that use first/done/next instead of front/empty/popFront, so I'm not sure what we're gaining here. An allocator instance is a variable like any other. So you use the classic techniques (shared globals, thread-local globals, passing around as parameter) for using the same allocator object from multiple places. Okay, that's fine... but this sort of manual management implies that I'm using it explicitly. That's where it all falls down for me. I think a disconnect here is that you think it where I think them. It's natural for an application to use one allocator that's not provided by the standard library, and it's often the case that an application defines and uses _several_ allocators for different parts of it. Then the natural question arises, how to deal with these allocators, pass them around, etc. etc. Eg, I want to use a library, it's allocation patterns are incompatible with my application; I need to provide it with an allocator. What now? Is every library responsible for presenting the user with a mechanism for providing allocators? What if the author forgets? (a problem I've frequently had to chase up in the past when dealing with 3rd party libraries) If the author forgets and hardcodes a library to use malloc(), I have no way around that. Once a library is designed to expect a user to supply an allocator, what happens if the user doesn't? Fall-back logic/boilerplate exists in every library I guess... The library wouldn't need to worry as there would be the notion of a default allocator (probably backed by the existing GC). And does that mean that applications+libraries are required to ALWAYS allocate through given allocator objects? Yes, they should. That effectively makes the new keyword redundant. new will still be used to tap into the global shared GC. std.allocator will provide other means of allocating memory. And what about the GC? The current global GC is unaffected for the time being. I can't really consider std.allocator intil it presents some usage patterns. Then you'd need to wait a little bit. It wasn't clear to me from your demonstration, but 'collect()' implies that GC becomes allocator-aware; how does that work? No, each allocator has its own means of dealing with memory. One could define a tracing allocator independent of the global GC. I'm not sure what this means. Other than I gather that the GC and allocators are fundamentally separate? Yes, they'd be distinct. Imagine an allocator that requests 4 MB from the GC as NO_SCAN memory, and then does its own management inside that block. User-level code allocates and frees e.g. strings or whatever from that block, without the global GC intervening. Is it possible to create a tracing allocator without language support? I think it is possible. Does the current language insert any runtime calls to support the GC? Aside from operator new, I don't think so. I want a ref-counting
Re: std.allocator needs your help
On 9/22/13 9:19 PM, Manu wrote: On 23 September 2013 13:01, Andrei Alexandrescu seewebsiteforem...@erdani.org mailto:seewebsiteforem...@erdani.org wrote: Containers and other objects that want to allow customization of their allocation would be parameterized during compilation with an allocator type. Functions that need to allocate memory may similarly accept a parameter of allocator type. You've just described C++ verbatim. And you've previously agreed that C++'s approach totally failed. Can you explain how this is different from C++, and how it solves the issues that defeated that design? As I just wrote in my previous post, the two mistakes in C++'s allocator design are: 1. Allocators are parameterized by type. 2. Allocators cannot have state. I fix both. One possibility I'm keeping in mind is that of defining a dynamic interface (i.e. in the OOP sense) for a global allocator. Then people can globally change what allocator must be used for operator new invocations. Right, this is getting warmer. It's a stark contrast to what you suggest above though, and when I try and think this through, it gets very complex, very fast. I can't imagine how such a system could ever be declared safe. However, this is more or less what I want. I don't know how to reconcile the 2 requirements. There are possibilities, which I hope to get to soon. How much have you thought on this? This is where I think some serious creativity will need to be applied... Following this train of thought, I can imagine a really nice end goal would be that the existing GC is effectively plugged in as a library, and people can easily substitute it for their own GC if they want/need to. Sean has already done that. The current GC is entirely replaceable (and in particular extricable). Andrei
Re: D2 is really that stable as it is claimed to be?
On 2013-09-21 21:47, Walter Bright wrote: That's gcc, and 4 is the line number (and the wrong line number) of the error. No column number. It's time you start using clang instead of gcc. -- /Jacob Carlborg
Re: std.allocator needs your help
On 24 September 2013 00:04, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 9/22/13 10:20 PM, Benjamin Thaut wrote: Am 23.09.2013 01:49, schrieb Andrei Alexandrescu: Hello, 2. Untyped allocator - traffics exclusively in ubyte[]. Why ubyte[] and not void[]? It's the logical choice at this level. ubyte[] == these are octets Isn't that what void[] also means? Except it says these are un-typed octets, ie, not a sequence of typed integers in the range 0-255.
Re: std.allocator needs your help
On 9/22/13 10:37 PM, Walter Bright wrote: On 9/22/2013 9:19 PM, Manu wrote: Following this train of thought, I can imagine a really nice end goal would be that the existing GC is effectively plugged in as a library, and people can easily substitute it for their own GC if they want/need to. It already is, and has been from the beginning. Rainer, for example, uses a different GC for VisualD. Correct. dmd knows naught about the GC. Well except for plugging a library call for calls to new. But that's expected. (I'm already pretending delete doesn't exist :o).) Andrei
Re: std.allocator needs your help
On 23 September 2013 23:38, Jacob Carlborg d...@me.com wrote: On 2013-09-23 11:31, qznc wrote: 5. Class local - The allocator is used for specific types (e.g. ASTNode in a compiler) 6. Class-hierarchy - The allocator is used for a specific type hierarchy (e.g. ASTNode might have sub classes Statement,BinOp,etc) 7. Container local - The C++ way which binds allocation to a wrapping container 8. Task local - like Thread local but for std.parallelism.Task That's it for now. This is quite a long list, which means it is probably exhaustive. There should be a generic approach which encompasses at least those cases. That's a good addition to the list. Another situation that I've encountered on a few occasions... Imagine I declare a particular allocator for my application, when dealing with 3rd party libs, I expect it to allocate within my application's heap. Seems like a situation where I might set a global allocator... Mr 3rd party library also has its own allocators though, things like pools or groups which it uses explicitly within it's code. I don't actually want to override these allocators, since they're effectively a higher-level construct, but I *do* want to override these allocator's memory source. Ie, they should allocate their pools from my application's heap, rather than the default heap. So in this way, it's important to be able to set overrides at multiple levels.
Re: std.allocator needs your help
On 9/22/13 10:20 PM, Benjamin Thaut wrote: Am 23.09.2013 01:49, schrieb Andrei Alexandrescu: Hello, 2. Untyped allocator - traffics exclusively in ubyte[]. Why ubyte[] and not void[]? It's the logical choice at this level. ubyte[] == these are octets Andrei
Re: D2 is really that stable as it is claimed to be?
On 2013-09-22 07:22, Walter Bright wrote: I'm not rejecting the idea outright. I've actually implemented this in the dmc compiler. It's just not terribly useful, and it has costs. Several people has asked for it and the LLVM community thinks it's worth it. -- /Jacob Carlborg
Re: std.allocator needs your help
On 9/23/13 1:03 AM, Dicebot wrote: On Sunday, 22 September 2013 at 23:49:56 UTC, Andrei Alexandrescu wrote: ... In general I like it though I do agree with concerns mentioned that without speculation about high-level API and usage examples it is rather hard to evaluate its practical applicability. For example, you have mentioned somewhere in comments that it is planned to embed allocator as part of template site like in C++ and it does not really sound a good idea. That is undecided at this point. It's possible that we collectively get convinced the efficiency impact of indirect calls is not too high, in which case we can have containers using a dynamic allocator interface. Andrei
Re: std.allocator needs your help
On 9/23/13 4:39 AM, Andrej Mitrovic wrote: On 9/23/13, Benjamin Thaut c...@benjamin-thaut.de wrote: Why ubyte[] and not void[]? Probably to make the GC avoid scanning into the array. No, that's determined at runtime by how the blocks are allocated. Andrei
Re: D2 is really that stable as it is claimed to be?
On 2013-09-22 04:38, Paul Jurczak wrote: That would do: 255 means column 256 or higher (256+). You would please more than 99.99% of users :-) We have lines with over 400 columns at work. Yes I know, horrible. -- /Jacob Carlborg
Re: std.allocator needs your help
On 9/23/13 12:31 AM, Jacob Carlborg wrote: On 2013-09-23 01:49, Andrei Alexandrescu wrote: I am making good progress on the design of std.allocator, and I am optimistic about the way it turns out. D's introspection capabilities really shine through, and in places the design does feel really archetypal - e.g. this is the essence of a freelist allocator. It's a very good feeling. The overall inspiration comes from Berger's HeapLayers, but D's introspection takes that pattern to a whole new level. I agree with Manu here. I thought the whole point was to come up with a design and API how the allocators are supposed to be used. How they integrate with user code. This is a disconnect. I say Well, I'm exploring car engines and this Otto cycle looks promising, here's how a transmission would work, and the brake system would be hydraulic... and you say but I want to know how to drive a car, and road regulations, and laws! Both are important, but they are very difficult to handle simultaneously in the same conversation. Andrei
Re: std.allocator needs your help
On 2013-09-23, 15:58, Andrei Alexandrescu wrote: I had imagined going into this that, like the range interface which the _language_ understands and interacts with, the allocator interface would be the same, ie, the language would understand this API and integrate it with 'new', and the GC... somehow. The D language has no idea what a range is. The notion is completely defined in std.range. The language has some knowledge of ranges, or foreach (e; myRange) would not work. -- Simen
Re: std.allocator needs your help
On 9/23/13 7:07 AM, Manu wrote: On 24 September 2013 00:04, Andrei Alexandrescu seewebsiteforem...@erdani.org mailto:seewebsiteforem...@erdani.org wrote: On 9/22/13 10:20 PM, Benjamin Thaut wrote: Am 23.09.2013 01:49, schrieb Andrei Alexandrescu: Hello, 2. Untyped allocator - traffics exclusively in ubyte[]. Why ubyte[] and not void[]? It's the logical choice at this level. ubyte[] == these are octets Isn't that what void[] also means? Except it says these are un-typed octets, ie, not a sequence of typed integers in the range 0-255. I think void[] means objects of unknown type. Andrei
Re: std.allocator needs your help
Great news! It looks like a big improvement on akward C++ allocators. (For what it's worth I have a working implementation of aligned malloc/free/realloc here https://github.com/p0nce/gfm/blob/master/gfm/core/memory.d, which can be the basis for an allocator layered upon another)
Re: std.allocator needs your help
On 9/23/13 7:22 AM, ponce wrote: Great news! It looks like a big improvement on akward C++ allocators. (For what it's worth I have a working implementation of aligned malloc/free/realloc here https://github.com/p0nce/gfm/blob/master/gfm/core/memory.d, which can be the basis for an allocator layered upon another) Awesome, thanks. Will look at it. Andrei
Re: std.allocator needs your help
On 24 September 2013 00:49, Andrei Alexandrescu seewebsiteforem...@erdan.org wrote: Simen Kjaeraas simen.kja...@gmail.com wrote: On 2013-09-23, 15:58, Andrei Alexandrescu wrote: I had imagined going into this that, like the range interface which the _language_ understands and interacts with, the allocator interface would be the same, ie, the language would understand this API and integrate it with 'new', and the GC... somehow. The D language has no idea what a range is. The notion is completely defined in std.range. The language has some knowledge of ranges, or foreach (e; myRange) would not work. Great point. I amend my claim. Mmm, precisely what I was talking about.
Re: [OT] Which IDE / Editor do you use?
On Mon, Sep 23, 2013 at 02:01:02PM +0200, Dicebot wrote: On Monday, 23 September 2013 at 11:52:28 UTC, Bruno Medeiros wrote: On 21/09/2013 16:07, Manu wrote: ... My feelings exactly. I learned about Linux and studied it when I was in high-school (Windows 98/Me era), and I was quite excited about it. Windows was more shite those days, and I knew Linux was not for the average user, but I thought that once I learned it well enough (shell, network, configuring partitions, automounting, the X server, etc.), it would be worthwhile to use. That wasn't the case unfortunately. There was always new stuff that would come up that you would need to learn how to configure, or need to thinker, or there would be shortcomings in application functionality. After a certain point it was just annoying. It might be fun for people who get kicks out of working the innards of a system and being closer to how things work, but on my computer I wanted to either have my leisure time, or get real work done. And spending time configuring stuff (that in Windows just worked out of the box) is not a productive use of one's time in any way, shape or form. True, this was like 10 years ago and Linux distros got better, but so has Windows, and nowdays there is little motivation now for me to try a different OS/desktop-environment. Ironically, this is exactly the reason I have never succeeded in using the Windows for daily work. Amount of manual configuration and subverting the defaults needed to make it actually usable for my programming flow is outstanding. In the same time on my Linux distro it is mostly `pacman -Sy gnome gnome-extra xorg-server nvidia dlang vim git` and I am ready to work on a fresh install. Ditto. I'm surprised at people talking about the amount of time spent configuring stuff on Linux, etc., because it's never happened to me! I mean, OK, in the early days you had to manually configure X11 and deal with all of the obscure problems, but that's no longer the case today. All I have to do is 'apt-get install package' and it Just Works(tm). I do like to tweak stuff -- and this is where Windows falls flat for me, 'cos it forces you to work a certain way, and when you go outside of that, things just stop working or it becomes an uphill battle (disclaimer: I haven't touched Windows for over a decade, so this may no longer be true today) -- but the default settings installed by apt-get *do* work. So I've no idea what people are talking about when they complain about needing to tweak this and that by hand. Linux *lets* you tweak stuff by hand, but, at least as far as Debian is concerned, the defaults pretty much Just Work. The whole bit about me recompiling the kernel and stuff -- the whole point was that I *wanted* to run a custom kernel, and Linux lets me do that. On Windows, this isn't even an option to begin with. The stock Debian kernel actually just works out of the box -- and had I wanted to, I could have just used that instead and never needed to do anything else. T -- INTEL = Only half of intelligence.
Re: std.allocator needs your help
On 9/23/13 8:02 AM, Adam D. Ruppe wrote: We should really deprecate the new keyword. It'd break like all code ever, but with D's templates, there's no need for it, and when it is there, it is going to spark problems about replacing global allocators or the library allocators being second class citizens. Maybe we could minimize the breakage by just rewriting the keyword into a library function call, which could then be overridden with local variables or imports via normal scoping. I'd think new already is translated into a library call. Walter? Andrei
Re: std.allocator needs your help
On 24 September 2013 00:05, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 9/22/13 10:37 PM, Walter Bright wrote: On 9/22/2013 9:19 PM, Manu wrote: Following this train of thought, I can imagine a really nice end goal would be that the existing GC is effectively plugged in as a library, and people can easily substitute it for their own GC if they want/need to. It already is, and has been from the beginning. Rainer, for example, uses a different GC for VisualD. Correct. dmd knows naught about the GC. Well except for plugging a library call for calls to new. But that's expected. (I'm already pretending delete doesn't exist :o).) delete is important if your class is being allocated by a pool or something... But you said before, people won't use 'new' if they are using an allocator. I'm really not sure that's a good idea. Most people (library authors!) don't actually care about their memory allocation, they will continue to use 'new', just because it's a keyword. It also screws with generic code; X should be allocated with 'new', but Y should be allocated with yAllocator.alloc()? What if you decide that Z, which allocates with 'new', becomes a problem and you want to switch it into a pool? You now need to track down every instance of 'new Z', and change it.
Re: std.allocator needs your help
We should really deprecate the new keyword. It'd break like all code ever, but with D's templates, there's no need for it, and when it is there, it is going to spark problems about replacing global allocators or the library allocators being second class citizens. Maybe we could minimize the breakage by just rewriting the keyword into a library function call, which could then be overridden with local variables or imports via normal scoping.
Re: std.allocator needs your help
Simen Kjaeraas simen.kja...@gmail.com wrote: On 2013-09-23, 15:58, Andrei Alexandrescu wrote: I had imagined going into this that, like the range interface which the _language_ understands and interacts with, the allocator interface would be the same, ie, the language would understand this API and integrate it with 'new', and the GC... somehow. The D language has no idea what a range is. The notion is completely defined in std.range. The language has some knowledge of ranges, or foreach (e; myRange) would not work. Great point. I amend my claim. Andrei
Re: std.allocator needs your help
On 9/23/13 7:50 AM, Manu wrote: delete is important if your class is being allocated by a pool or something... It's important as an API function, but not as a language primitive. new should also have been a function. But you said before, people won't use 'new' if they are using an allocator. I'm really not sure that's a good idea. I don't see another way if people are to define and use multiple allocators. Most people (library authors!) don't actually care about their memory allocation, they will continue to use 'new', just because it's a keyword. new considered harmful etc. It also screws with generic code; X should be allocated with 'new', but Y should be allocated with yAllocator.alloc()? What if you decide that Z, which allocates with 'new', becomes a problem and you want to switch it into a pool? You now need to track down every instance of 'new Z', and change it. We can only improve that situation by allowing people to replace the global allocator with their own allocators. Again, there's a disconnect here - I'm discussing how to make it easy for people to define allocators and you discuss how to make it possible for people to plug allocators, once defined, as the global allocator. These are two distinct endeavors. At the level I'm at, I'm concerned with making good allocators easy to implement. You may say you don't care, and that's good feedback, but it's what I have for the time being. Andrei
Re: std.allocator needs your help
On 9/23/13 7:22 AM, ponce wrote: Great news! It looks like a big improvement on akward C++ allocators. (For what it's worth I have a working implementation of aligned malloc/free/realloc here https://github.com/p0nce/gfm/blob/master/gfm/core/memory.d, which can be the basis for an allocator layered upon another) I gave this a read, nice work. One question: what circumstances require run-time alignment values, and what values would those be? I'm currently under the assumption that alignments are known during compilation. Thanks, Andrei
Re: std.allocator needs your help
On 23 September 2013 23:58, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 9/22/13 9:03 PM, Manu wrote: On 23 September 2013 12:28, Andrei Alexandrescu seewebsiteforem...@erdani.org mailto:SeeWebsiteForEmail@**erdani.orgseewebsiteforem...@erdani.org wrote: My design makes it very easy to experiment by allowing one to define complex allocators out of a few simple building blocks. It is not a general-purpose allocator, but it allows one to define any number of such. Oh okay, so this isn't really intended as a system then, so much a suggested API? For some definition of system and API, yes :o). That makes almost all my questions redundant. I'm interested in the system, not the API of a single allocator (although your API looks fine to me). I already have allocators I use in my own code. Naturally, they don't inter-operate with anything, and that's what I thought std.allocator was meant to address. Great. Do you have a couple of nontrivial allocators (heap, buddy system etc) that could be adapted to the described API? Err, not really actually. When I use custom allocator's, it's for performance, which basically implies that it IS a trivial allocator :) The common ones I use are: stack-based markrelease, circular buffers, pools, pool groups (collection of different sized pools)... that might be it actually. Very simple tools for different purposes. The proposed design makes it easy to create allocator objects. How they are used and combined is left to the application. Is that the intended limit of std.allocator's responsibility, or will patterns come later? Some higher level design will come later. I'm not sure whether or not you'll find it satisfying, for reasons I'll expand on below. Leaving the usage up to the application means we've gained nothing. I already have more than enough allocators which I use throughout my code. The problem is that they don't inter-operate, and certainly not with foreign code/libraries. This is what I hoped std.allocator would address. Again, if you already have many allocators, please let me know if you can share some. std.allocator will prescribe a standard for defining allocators, with which the rest of std will work, same as std.range prescribes a standard for defining ranges, with which std.algorithm, std.format, and other modules work. Clearly one could come back with but I already have my own ranges that use first/done/next instead of front/empty/popFront, so I'm not sure what we're gaining here. No, it's just that I'm saying std.allocator needs to do a lot more than define a contract before I can start to consider if it solves my problems. This is a good first step though, I'm happy to discuss this, but I think discussion about the practical application may also reveal design details at this level. It's like you say, I can rename my allocator's methods to suit an agreed standard, that'll take me 2 minutes, but it's how the rest of the universe interacts with that API that matters, and if it effectively solves my problems. An allocator instance is a variable like any other. So you use the classic techniques (shared globals, thread-local globals, passing around as parameter) for using the same allocator object from multiple places. Okay, that's fine... but this sort of manual management implies that I'm using it explicitly. That's where it all falls down for me. I think a disconnect here is that you think it where I think them. It's natural for an application to use one allocator that's not provided by the standard library, and it's often the case that an application defines and uses _several_ allocators for different parts of it. Then the natural question arises, how to deal with these allocators, pass them around, etc. etc. No, I certainly understand you mean 'them', but you lead to what I'm asking, how do these things get carried/passed around. Are they discreet, or will they invade argument lists everywhere? Are they free to flow in/out of libraries in a natural way? These patterns are what will define the system as I see it. Perhaps more importantly, where do these allocators get their memory themselves (if they're not a bottom-level allocator)? Global override perhaps, or should a memory source always be explicitly provided to a non-bottom-level allocator? Eg, I want to use a library, it's allocation patterns are incompatible with my application; I need to provide it with an allocator. What now? Is every library responsible for presenting the user with a mechanism for providing allocators? What if the author forgets? (a problem I've frequently had to chase up in the past when dealing with 3rd party libraries) If the author forgets and hardcodes a library to use malloc(), I have no way around that. Sure, but the common case is that the author will almost certainly use keyword 'new'. How can I affect that as a 3rd
Re: std.allocator needs your help
On 24 September 2013 01:02, Andrei Alexandrescu seewebsiteforem...@erdani.org wrote: On 9/23/13 7:50 AM, Manu wrote: delete is important if your class is being allocated by a pool or something... It's important as an API function, but not as a language primitive. new should also have been a function. I like operator new. It goes blue, and the statement isn't riddled with exclamation marks. Imagine if new was a template (it would have to be a template). Allocating a template would require nested template syntax every time, which is pretty ugly. X x = new!(X!arg(args...)); // ewoo, paren spam... But you said before, people won't use 'new' if they are using an allocator. I'm really not sure that's a good idea. I don't see another way if people are to define and use multiple allocators. Well if an allocator is somehow associated with a class/struct, then 'new MyClass' would invoke that allocator. If you: pushThreadAllocator(myAllocator); X x = new X; popThreadAllocator(); Then new will use your thread-local allocator. Continue for global, fiber, etc. Maybe there's opportunity for 'scope' to offer some nice convenience here? Most people (library authors!) don't actually care about their memory allocation, they will continue to use 'new', just because it's a keyword. new considered harmful etc. It also screws with generic code; X should be allocated with 'new', but Y should be allocated with yAllocator.alloc()? What if you decide that Z, which allocates with 'new', becomes a problem and you want to switch it into a pool? You now need to track down every instance of 'new Z', and change it. We can only improve that situation by allowing people to replace the global allocator with their own allocators. Again, there's a disconnect here - I'm discussing how to make it easy for people to define allocators and you discuss how to make it possible for people to plug allocators, once defined, as the global allocator. These are two distinct endeavors. At the level I'm at, I'm concerned with making good allocators easy to implement. You may say you don't care, and that's good feedback, but it's what I have for the time being. Well, I'm discussing how do people affect/override 'new' in various circumstances? But sure, I said before, if we're limiting this discussion to the API of an allocator then it looks fine, I see no obvious issues. But I think the question requires consideration of the intended goal to make informed decisions about the design, even at this level.
Re: std.allocator needs your help
On Monday, 23 September 2013 at 15:45:25 UTC, Andrei Alexandrescu wrote: On 9/23/13 7:22 AM, ponce wrote: Great news! It looks like a big improvement on akward C++ allocators. (For what it's worth I have a working implementation of aligned malloc/free/realloc here https://github.com/p0nce/gfm/blob/master/gfm/core/memory.d, which can be the basis for an allocator layered upon another) I gave this a read, nice work. One question: what circumstances require run-time alignment values, and what values would those be? I'm currently under the assumption that alignments are known during compilation. Thanks, Andrei I don't know of a use case for run-time alignment values.
Re: std.allocator needs your help
On 9/23/13 9:47 AM, Benjamin Thaut wrote: I always understood void[] as block of unkown data. Which a allocator should return in my opinion. Whats the point of void having a size in D if we still do it the C way? In my opinion ubyte[] is a array of values in the range of 0-255 like manu says. Also if you get a ubyte[] you might get the opinion that it is initialized to all zeros or something. Which might not be true for all allocators (performance ...) If you get a void[] you know, all bets are off, and you have to check if the allocator preinitialized it or not. You might be right. For example, ubyte[] allows arithmetic on its elements, which is something one shouldn't ever care to do in an allocation library. I'm unclear on what void[] means starting from its semantics. That said, I replaced ubyte[] with void[] throughout my existing code and it works. Andrei
Re: std.allocator needs your help
On Sunday, 22 September 2013 at 23:49:56 UTC, Andrei Alexandrescu wrote: Hello, First of all, awesome ! Now the mh part. I really think the it thing is not good. I don't think it is desirable or necessary. We should get rid of it. You can't deal with ubyte[] like that, that is incorrect in regard to - unimplemented - aliasing rules. Allocator should deal with void[] . What you call safe really isn't. Allocate something on the GC, store a pointer on a custom allocated location, collect, enjoy the memory corruption. All operation are safe according to your proposal. Allocation can only be safe if the GRAND MASTER GC is aware of it. You proposal allocate shared memory. This is common in C/C++ world as memory is shared by default, but shouldn't be in D. It is highly desirable to allocate with different methods for different type qualifier. How does your design adapt to that ? Finally, we got to decide how these basics block are used to form typed allocators, and interact with language constructs. Sorry if this has been mentioned before, it is really ate here and I can't read the whole thread, especially since Manu is on steroids :D
Re: std.allocator needs your help
Am Tue, 24 Sep 2013 00:50:02 +1000 schrieb Manu turkey...@gmail.com: [...] It also screws with generic code; X should be allocated with 'new', but Y should be allocated with yAllocator.alloc()? What if you decide that Z, which allocates with 'new', becomes a problem and you want to switch it into a pool? You now need to track down every instance of 'new Z', and change it. That's not a problem with the proposed GC allocator. We should add proper overloads to emplace and a create function and then all code should use create!(Allocator, MyClass)(args) or create!MyClass(allocatorInstance, args). New should then be considered as deprecated and replaced by create!(GCAllocator, Class)().
Re: std.allocator needs your help
On 9/23/13, Johannes Pfau nos...@example.com wrote: New should then be considered as deprecated and replaced by create!(GCAllocator, Class)(). What? That's never gonna happen. For one thing, it looks ugly as hell. And for another, it's going to break everything written in D.
Re: std.allocator needs your help
Am 23.09.2013 16:16, schrieb Andrei Alexandrescu: On 9/23/13 7:07 AM, Manu wrote: On 24 September 2013 00:04, Andrei Alexandrescu seewebsiteforem...@erdani.org mailto:seewebsiteforem...@erdani.org wrote: On 9/22/13 10:20 PM, Benjamin Thaut wrote: Am 23.09.2013 01:49, schrieb Andrei Alexandrescu: Hello, 2. Untyped allocator - traffics exclusively in ubyte[]. Why ubyte[] and not void[]? It's the logical choice at this level. ubyte[] == these are octets Isn't that what void[] also means? Except it says these are un-typed octets, ie, not a sequence of typed integers in the range 0-255. I think void[] means objects of unknown type. Andrei I always understood void[] as block of unkown data. Which a allocator should return in my opinion. Whats the point of void having a size in D if we still do it the C way? In my opinion ubyte[] is a array of values in the range of 0-255 like manu says. Also if you get a ubyte[] you might get the opinion that it is initialized to all zeros or something. Which might not be true for all allocators (performance ...) If you get a void[] you know, all bets are off, and you have to check if the allocator preinitialized it or not. Kind Regards Benjamin Thaut
Re: std.allocator needs your help
Am Mon, 23 Sep 2013 18:36:57 +0200
schrieb Andrej Mitrovic andrej.mitrov...@gmail.com:
On 9/23/13, Johannes Pfau nos...@example.com wrote:
New should then be considered as deprecated and replaced by
create!(GCAllocator, Class)().
What? That's never gonna happen. For one thing, it looks ugly as hell.
And for another, it's going to break everything written in D.
That's why I say considered and not really deprecated. But if you
want to / have to write allocator aware code which can use the GC it's a
nice solution:
auto list(Payload, Allocator = GCAllocator)()
{
create!(Allocator) ...
}
Of course the API should be improved.
For example create could default to the GC allocator. Then it's
auto a = new Class(...) vs auto a = create!Class(...).
But IIRC when emplace was first implemented and class allocators were
removed it was quite clear that new could be easily replaced by a
template function and has no real place as a builtin anymore. It's just
too late to remove it.
Re: std.allocator needs your help
Am 23.09.2013 17:02, schrieb Adam D. Ruppe:
We should really deprecate the new keyword. It'd break like all code
ever, but with D's templates, there's no need for it, and when it is
there, it is going to spark problems about replacing global allocators
or the library allocators being second class citizens.
Its not possible to replace new with a library function in all cases.
There are many examples where it breaks (I tried it believe me). Just
let me give a few:
class A
{
class B
{
}
B m_b;
this()
{
// uses the sourrounding context
// not possible with library function
m_b = new B();
}
}
Also there is the problem that D does not have perfect forwarding. That
means if new is a library function it will cause additional copies /
moves of structs which are not the case with the buildin new. Next there
are implict conversions of literals. Those work just fine with the
buildin new, but will break with a templated library new.
E.g.
class C
{
this(ubyte a, ubyte b)
{
}
}
new C(5,3); // fine
New!C(5,3); // Error: can not implicitly convert int to ubyte
Unless of course you want to write a template metaprogram that does
those implict conversions. Although the problem would remain that you
can't know if the value you get comes from a literal or from a function
call, variable, etc.
The next thing are arrays. While you get the nice array syntax with the
buildin new, a library new just looks ugly.
new int[5];
vs.
NewArray!int(5); // Can't use New! here because it would conflict with
the New! for creating objects / structs
I'm using library implemented new / delete since over a year and its
annoying and makes the language look ugly and feel strange to use. See
my allocator and New / Delete implementation here:
https://github.com/Ingrater/druntime/blob/master/src/core/allocator.d
I would rather want new to be overloadable and have 2 sets of parameters
new (allocator)(arg1, arg2)
Where allocator would go to the overloaded version of new and arg1
and arg2 will be passed to the constructor.
I think its a really bad idea to deprecate new.
Replacing Delete works just fine with a library function though.
Kind Regards
Benjamin Thaut
Re: std.allocator needs your help
On 9/23/13 10:01 AM, deadalnix wrote: On Sunday, 22 September 2013 at 23:49:56 UTC, Andrei Alexandrescu wrote: Hello, First of all, awesome ! Now the mh part. I really think the it thing is not good. I don't think it is desirable or necessary. We should get rid of it. The singleton allocator it is instrumental for supporting stateful and stateless allocators with ease. It took me a few iterations to get to that, and I'm very pleased with the results. I think it would be difficult to improve on it without making other parts more difficult. You can't deal with ubyte[] like that, that is incorrect in regard to - unimplemented - aliasing rules. Allocator should deal with void[] . I think I'll do that. What you call safe really isn't. Allocate something on the GC, store a pointer on a custom allocated location, collect, enjoy the memory corruption. I don't understand this. There are no pointers at this level, only untyped memory. The main chance of corruption is to access something after it's been freed. All operation are safe according to your proposal. No, for most allocators freeing and reallocating are unsafe. Allocation can only be safe if the GRAND MASTER GC is aware of it. I don't think so. You proposal allocate shared memory. No. It allocates unshared memory off of a shared or unshared allocator. The memory just allocated is as of yet unshared for the simple reason that only one thread has as of yet access to it. This is common in C/C++ world as memory is shared by default, but shouldn't be in D. It is highly desirable to allocate with different methods for different type qualifier. How does your design adapt to that ? The typed allocators will have distinct methods for shared and unshared allocation. Even at this level it's possible to design an allocator that allocates in different ways with a shared vs. an unshared allocator object (overload on shared). So far I've only designed non-shared allocators, or wrap those that are already shared (malloc and new). Finally, we got to decide how these basics block are used to form typed allocators, and interact with language constructs. Sure. Again: I describe the Otto cycle and you discuss how to paint the road. Andrei
Re: std.allocator needs your help
On 9/23/13 8:32 AM, Manu wrote: On 23 September 2013 23:58, Andrei Alexandrescu seewebsiteforem...@erdani.org mailto:seewebsiteforem...@erdani.org wrote: This is a good first step though, I'm happy to discuss this, but I think discussion about the practical application may also reveal design details at this level. Absolutely. This is refreshing since I've gone through the cycle of types must be integrated into allocators and with the global GC ... - ... untyped allocators can actually be extricated once. It is now great to revisit the assumptions again. One matter I'm rather surprised didn't come up (I assumed everyone would be quite curious about it) is that the allocator does not store the allocated size, or at least does not allow it to be discovered easily. This is a definitely appropriate topic for the design at hand. The malloc-style APIs go like: void* malloc(size_t size); void free(void*); Note that the user doesn't pass the size to free(), which means the allocator is burdened with inferring the size of the block from the pointer efficiently. Given that most allocators make crucial strategic choices depending on the size requested, this API shortcoming is a bane of allocator writers everywhere, and a source of inefficiency in virtually all contemporary malloc-style allocators. This is most ironic since there is next to nothing that user code can do with a pointer without knowing the extent of memory addressed by it. (A notable exception is zero-terminated strings, another questionable design decision.) It all harkens back to Walter's claim of C's biggest mistake (with which I agree) of not defining a type for a bounded memory region. Upon studying a few extant allocators and D's lore, I decided that in D things have evolved sufficiently to have the user pass the size upon deallocation as well: void[] allocate(size_t size); void deallocate(void[] buffer); This is because the size of D objects is naturally known: classes have it in the classinfo, slices store it, and the few cases of using bald pointers for allocation are irrelevant and unrecommended. This all makes memory allocation in D a problem fundamentally simpler than in C, which is quite an interesting turn :o). No, I certainly understand you mean 'them', but you lead to what I'm asking, how do these things get carried/passed around. Are they discreet, or will they invade argument lists everywhere? Since there's a notion of the default allocator, there will be ways to push/pop user-defined allocators that temporarily (or permanently) replace the default allocator. This stage of the design is concerned with allowing users to define such user-defined allocators without having to implement them from scratch. Are they free to flow in/out of libraries in a natural way? These patterns are what will define the system as I see it. Perhaps more importantly, where do these allocators get their memory themselves (if they're not a bottom-level allocator)? Global override perhaps, or should a memory source always be explicitly provided to a non-bottom-level allocator? There are a few obvious bottom allocators, such as malloc, new, sbrk, or Windows heap. All other allocators will compose by parameterizing on their parent allocator. Sure, but the common case is that the author will almost certainly use keyword 'new'. How can I affect that as a 3rd party? This would require me overriding the global allocator somehow... which you touched on earlier. The way I'm thinking of this is to install/uninstall user-defined allocators that will satisfy calls to new. Since not all allocators support tracing, some would require the user to deallocate stuff manually. The library wouldn't need to worry as there would be the notion of a default allocator (probably backed by the existing GC). Right. So it's looking like like the ability to override the global allocator is a critical requirement. Again, this is difficult to handle in the same conversation as should we pass size upon deallocation. Yes, we need road laws, but it's hard to talk about those and engine lubrication in the same breath. For me, critical right now is to assess whether the untyped API misses an important category of allocators, what safety level it has (that's why expand is so different from realloc!) etc. And does that mean that applications+libraries are required to ALWAYS allocate through given allocator objects? Yes, they should. Then we make keyword 'new' redundant? Probably not. Typed allocators will need to integrate with (and occasionally replace) the global shared GC. The problem is that installing an allocator does not get to define what a pointer is and what a reference is. Why not? Because that requires a language change. I'm not sure you realize but you move the goalposts all the time. We were talking within the context of libraries and installing
Re: D2 is really that stable as it is claimed to be?
On Sep 23, 2013 6:30 PM, Sean Kelly s...@invisibleduck.org wrote: On Sep 21, 2013, at 10:22 PM, Walter Bright newshou...@digitalmars.com wrote: On 9/21/2013 8:54 PM, Michel Fortin wrote: I don't think it should be a priority, but rejecting the idea outright is shortsighted in my opinion. I'm not rejecting the idea outright. I've actually implemented this in the dmc compiler. It's just not terribly useful, and it has costs. I'd consider it in a similar class as the dictionary lookup that occurs when an unknown symbol is encountered. Totally unnecessary, but it's a nice time-saver. Is it clang that displays the line in error with a carat underneath the error? Though if there really isn't an efficient way to do it in DMD then I don't think it's worthwhile. I was only thinking of the parser when I mentioned the beginning-of-line pointer. I hadn't considered the AST. GCC has a carat too now. Regards -- Iain Buclaw *(p e ? p++ : p) = (c 0x0f) + '0';
Re: D2 is really that stable as it is claimed to be?
On Sep 21, 2013, at 10:22 PM, Walter Bright newshou...@digitalmars.com wrote: On 9/21/2013 8:54 PM, Michel Fortin wrote: I don't think it should be a priority, but rejecting the idea outright is shortsighted in my opinion. I'm not rejecting the idea outright. I've actually implemented this in the dmc compiler. It's just not terribly useful, and it has costs. I'd consider it in a similar class as the dictionary lookup that occurs when an unknown symbol is encountered. Totally unnecessary, but it's a nice time-saver. Is it clang that displays the line in error with a carat underneath the error? Though if there really isn't an efficient way to do it in DMD then I don't think it's worthwhile. I was only thinking of the parser when I mentioned the beginning-of-line pointer. I hadn't considered the AST.
Re: std.allocator needs your help
On Monday, 23 September 2013 at 17:01:47 UTC, deadalnix wrote: What you call safe really isn't. Allocate something on the GC, store a pointer on a custom allocated location, collect, enjoy the memory corruption. All operation are safe according to your proposal. Allocation can only be safe if the GRAND MASTER GC is aware of it. I don't see why the GC can't be allocator-agnostic. In principle, it should be possible to register arbitrary allocators with the GC (possibly by storing a pointer to the allocator in each GC-managed block of memory). The GC would have a default allocator which would be used with the new operator, and something like the aforementioned create!() syntax could be used for custom allocators: SomeClass c1 = new SomeClass(arg); //Managed by the GC SomeClass c2 = gcCreate!SomeClass(someAllocator, arg); //Another possible syntax (much prettier!): SomeClass c3 = someAllocator.new SomeClass(arg); This could also integrate with DIP46; instead of pushing and popping a GC object, one would push and pop the GC's default allocator (for the current thread, of course). There would be a bit of overhead, but that seems to be unavoidable in a sufficiently generic design, and I doubt that it would cut performance to an unacceptable level, especially because really performance-critical code would avoid the GC anyway. You proposal allocate shared memory. This is common in C/C++ world as memory is shared by default, but shouldn't be in D. It is highly desirable to allocate with different methods for different type qualifier. How does your design adapt to that ? If the aforementioned thread-local GC reference is implemented, it should be possible to provide decent thread-local allocation. I'm not sure exactly how to design that, but it seems like thread-local GC allocation would require some work at the GC level; a global GC won't necessarily play nicely with thread-local allocators. For non-GC thread-local allocations, I'd imagine that you could just create a thread-local allocator.
Re: std.allocator needs your help
On 9/23/2013 3:17 AM, Manu wrote: Sure, true. But little things like separate it into its own lib, so you can easily link a different one, or not link one at all. To link in a different one, put the different one on the link command before libphobos2.a. The different one will override the one in the library. Also, what about language support? How it's done is deliberately not part of the language. How much language support is there currently? Zero. I don't think I could currently write the ref-counting GC that I'd like without extensive language support. inc/dec ref calls would need to be inserted all over the place by he compiler, and optimiser would need to know how to remove redundant inc/dec sequences. A ref counting system is something entirely different than just plugging in another GC.
Re: std.allocator needs your help
On Monday, 23 September 2013 at 13:58:42 UTC, Andrei Alexandrescu wrote: If the author forgets and hardcodes a library to use malloc(), I have no way around that. We should consider the lib writer use new and/or automatic allocation like for closures. In practice, this is what will happen. new will still be used to tap into the global shared GC. std.allocator will provide other means of allocating memory. That uterly suck and we can do better using type informations. At this point collect() is only implemented by the global GC. It is possible I'll drop it from the final design. However, it's also possible that collect() will be properly defined as collect all objects allocated within this particular allocator that are not referred from any objects also allocated within this allocator. I think that's a useful definition. I don't think collect is usefull with no type information (not even if the data may contain pointers). The D language has no idea what a range is. The notion is completely defined in std.range. foreach has some idea of what a range is.
Re: compiled code file size
On Sep 21, 2013, at 8:49 AM, Manu turkey...@gmail.com wrote: On 21 September 2013 21:34, Temtaime temta...@gmail.com wrote: Are you saying about passing a function via pointer to winapi for example? The logic is simple: if someone gets function address, then function cannot be stripped. It's logic of all c++ compilers. Totally OT, but every single time I read your name when you post, I can't help but start hearing lines from Terry Prachett's Hogfather in my head… Same here.
Re: compiled code file size
On Sat, Sep 21, 2013 at 09:41:30PM +0200, Vladimir Panteleev wrote: On Friday, 20 September 2013 at 16:20:34 UTC, Duke Normandin wrote: I'm re-visiting the D language. I've compared the file sizes of 2 executables - 1 is compiled C code using gcc; the other is D code using dmd. helloWorld.d = helloWorld.exe = 146,972 bytes ex1hello.c = ex1-hello.exe = 5,661 bytes Why such a huge difference??? You can upload a .map file here, and see what's taking up all the space: http://thecybershadow.net/d/mapview/ Ah, you beat me to it. :-) T -- All problems are easy in retrospect.
Re: D2 is really that stable as it is claimed to be?
Regarding the improving the error messages of D compilers, I think having the column number is nice, but there are two improvements that I think are more important: 1) One of them is the aka, that is showing both the name of aliases and the aliased types/values: http://d.puremagic.com/issues/show_bug.cgi?id=5004 2) The other improvement I'd like for D error messages and warnings is to give a standard error number. This is a simple improvement, but it makes simpler to write explanation pages for the errors. The C# compiler and other compilers have them. Bye, bearophile
Re: [OT] Which IDE / Editor do you use?
On 23/09/2013 15:50, H. S. Teoh wrote: I'm surprised at people talking about the amount of time spent configuring stuff on Linux, etc., because it's never happened to me! I mean, OK, in the early days you had to manually configure X11 and deal with all of the obscure problems, but that's no longer the case today. All I have to do is 'apt-get install package' and it Just Works(tm). Yeah, that was several years ago. Nowadays the hardware support and configuration seems to work straight out from the box as well. The remaining reasons that keep me away from Linux are other now. -- Bruno Medeiros - Software Engineer
Re: std.allocator needs your help
On 2013-09-23 17:47, Manu wrote: I like operator new. It goes blue, and the statement isn't riddled with exclamation marks. For Ruby, many editors and IDE's highlight standard built in functions as keywords. Like new, require, attr_accessor (and friends) to mention a few. Imagine if new was a template (it would have to be a template). Allocating a template would require nested template syntax every time, which is pretty ugly. X x = new!(X!arg(args...)); // ewoo, paren spam... Why not: X x = new!X(args); -- /Jacob Carlborg
Re: std.allocator needs your help
On 09/23/2013 07:16 PM, Andrei Alexandrescu wrote: On 9/23/13 10:01 AM, deadalnix wrote: Finally, we got to decide how these basics block are used to form typed allocators, and interact with language constructs. Sure. Again: I describe the Otto cycle and you discuss how to paint the road. IMO there is a problem with this metaphor. If the road is painted metallic the Otto motor (cycle) needs cooling, otherwise ... boom. So I think the overall picture has to be kept in sight.
Re: std.allocator needs your help
On 2013-09-23 20:59, Jacob Carlborg wrote: http://wiki.dlang.org/Runtime_Hooks Search for new. -- /Jacob Carlborg
Re: std.allocator needs your help
On 2013-09-23 16:05, Manu wrote: Another situation that I've encountered on a few occasions... Imagine I declare a particular allocator for my application, when dealing with 3rd party libs, I expect it to allocate within my application's heap. Seems like a situation where I might set a global allocator... Mr 3rd party library also has its own allocators though, things like pools or groups which it uses explicitly within it's code. I don't actually want to override these allocators, since they're effectively a higher-level construct, but I *do* want to override these allocator's memory source. Ie, they should allocate their pools from my application's heap, rather than the default heap. So in this way, it's important to be able to set overrides at multiple levels. You might be able to combine two different allocators. Basically create a new allocator that somehow extends the third party allocator but allocates in your own heap instead. -- /Jacob Carlborg
Re: std.allocator needs your help
On 2013-09-23 17:03, Andrei Alexandrescu wrote: I'd think new already is translated into a library call. Walter? Yes, new is lowered to a couple different runtime functions. Here's a list: http://wiki.dlang.org/Runtime_Hooks -- /Jacob Carlborg
Re: D2 is really that stable as it is claimed to be?
On 9/23/13, Andrej Mitrovic andrej.mitrov...@gmail.com wrote: I have a partial implementation of this in one of my branches Also, I will not continue work on this unless Walter greenlights it. I'm not going to put any work that's inevitably going to be thrown away.
Re: std.allocator needs your help
On 9/23/13 11:24 AM, Walter Bright wrote: On 9/23/2013 3:17 AM, Manu wrote: Sure, true. But little things like separate it into its own lib, so you can easily link a different one, or not link one at all. To link in a different one, put the different one on the link command before libphobos2.a. The different one will override the one in the library. Also, what about language support? How it's done is deliberately not part of the language. How much language support is there currently? Zero. I think there's at least indirect support in e.g. generating typeinfo objects appropriately. Many of the components of those typeinfos are expressly defined for helping a tracing collector. Andrei
Re: D2 is really that stable as it is claimed to be?
On 9/23/13, bearophile bearophileh...@lycos.com wrote: 1) One of them is the aka, that is showing both the name of aliases and the aliased types/values: http://d.puremagic.com/issues/show_bug.cgi?id=5004 I have a partial implementation of this in one of my branches, but IIRC it was difficult do cover all cases since the compiler internally inserts a bunch of aliases as well. Those could be marked that they're internal, so that's fixable. Another issue I ran into is that diagnostics are called after the retrieval of the aliased-to type, which basically means by the time the compiler issues errors the alias declaration is gone, the compiler only works on the target symbol. I had a workaround for this, but it's going to take more work to get done.
Re: D2 is really that stable as it is claimed to be?
On 2013-09-21 23:12, Andrei Alexandrescu wrote: I'm ambivalent because the matter is fuzzy. It is factually true that new releases will break code. On the other hand, that is the case with most compiler releases even for mature languages (at Facebook upgrading across minor gcc releases _always_ entails significant disruption). On the third hand (sic), there are companies and projects using D in the real world so stating that is unstable would do little else than either shoo people away for no good reason. Some of these companies still use D1. About the code breakage. I think it's still an issue that bugfixes and language changes occur in the same release. No notation of major, minor and patch releases. -- /Jacob Carlborg
Re: std.allocator needs your help
On 9/23/13 11:59 AM, Jacob Carlborg wrote: On 2013-09-23 17:03, Andrei Alexandrescu wrote: I'd think new already is translated into a library call. Walter? Yes, new is lowered to a couple different runtime functions. Here's a list: http://wiki.dlang.org/Runtime_Hooks Thanks, this is great. Currently new is fail because it calls functions passing TypeInfo objects, instead of calling type-parameterized functions. We must change that to have new T(args) lower into .opNew!T(args). Then object.d defines that function. Andrei
Re: std.allocator needs your help
On 9/23/13 11:55 AM, Robert Schadek wrote: On 09/23/2013 07:16 PM, Andrei Alexandrescu wrote: On 9/23/13 10:01 AM, deadalnix wrote: Finally, we got to decide how these basics block are used to form typed allocators, and interact with language constructs. Sure. Again: I describe the Otto cycle and you discuss how to paint the road. IMO there is a problem with this metaphor. If the road is painted metallic the Otto motor (cycle) needs cooling, otherwise ... boom. So I think the overall picture has to be kept in sight. Which makes the metaphor unintentionally better. Yes, we do need to worry about higher concerns because that's how layerd abstractions work. Andrei
Re: Download page needs a tidy up
On Thursday, 22 August 2013 at 13:14:01 UTC, Ludovit Lucenic wrote: On Thursday, 22 August 2013 at 03:23:57 UTC, Manu wrote: I got the latest GDC from dlang.org/download, but it doesn't work. libiconv-2.dll (and possibly others?) is not included in the package. If this depends on another package being present, there is no text anywhere along the download path to inform the user. On 22 August 2013 13:07, Manu turkey...@gmail.com wrote: So I'm trying to find windows binaries for GDC and LDC... First place I look is dlang.org/download. Appears to be for DMD... keep looking. I look at the GDC/LDC wiki pages. No links to binaries anywhere. GDC and LDC home pages... no links to binaries. Github doesn't host binaries anymore... Where are they? Turns out there are links to the GDC binaries (hosted on bitbucket) on dlang.org/download. ...I didn't previously notice they were there, never scrolled down far enough. The impression you get from the top of the page is that dlang.orgis just DMD related, and I quickly dismissed it previously _ But there's still no LDC binary there... where is it? This needs to be fixed. You can argue I'm retarded and ignorant, but as an end user, it should take me no more than 5 seconds to find the download button. I suggest, on the front page of dlang.org, there should be a MASSIVE button: DOWNLOAD D COMPILERS, and the download page should be tweaked to be more obviously compiler agnostic. D1 and DMC consume an unreasonable amount of realestate, hiding GDC/LDC (surely basically nobody is looking for those?), perhaps they should be reduced to small test links with the other links down the bottom of the page? This will allow room to present GDC and LDC without scrolling. And why is there no LDC binary? I am completely with you, Manu. Same with me with GDC under Windows. Missing libiconv-2.dll... The same happened to me when looking around for the other two D compilers. It took me a couple of days until I found the link on the dlang.org's download page. Any suggestions on where to get a working copy of GDC to run on Windows, please? Thanks, Ludovit I want to try gdc and I get the same error on libiconv-2.dll
Re: D2 is really that stable as it is claimed to be?
On 09/22/2013 09:27 PM, Walter Bright wrote:
On 9/22/2013 11:43 AM, Timon Gehr wrote:
Tracking line numbers is likely worth it. I don't believe that
providing column
numbers in error messages necessitates a slowdown though.
Please consider that:
IT ISN'T JUST FOR ERROR MESSAGES
It would go in the symbolic debug info, too, where it will be required
everywhere and will be right there on the fast path through the
lexer/compiler.
...
There is no such thing as a law that obliges compiler writers to add
column numbers in debug info when such information is available in
frontend error messages. The trade-offs involved in both cases may be
different and deserve separate consideration.
Now consider the lexer doing a fast skip over comment text (this ranks
fairly high in the profile). This operation gets a lot slower if you're
also keeping track of column number.
I am not keeping track of column number.
Please note that:
COLUMN NUMBER ISN'T THE OFFSET FROM THE START OF THE LINE
...
Obviously. I compute the correct column number exactly in the case when
an error message should actually be printed. It is not necessary to do
any of this on the fast path. The additional memory word per location
that I waste in comparison to DMD could be shaved off by using more
computation in the error case (or by giving up support for exact
underlining), but the project has not yet reached a stage where this is
worth considering/measuring.
Excerpts from actual code I wrote roughly two years ago:
class Source{
// computes a slice of the entire first line
// where some given slice occurs in the source buffer.
// this allows to recover column information on the fly, and we
// will also be able to print the line where an error occurred
// without storing it explicitly.
// running time is linear in output length
string getLineOf(string rep)in{/*...*/}out{/*...*/}body{
string before=code[0..rep.ptr-code.ptr];
string after=code[rep.ptr-code.ptr..$];
immutable(char)* start=code.ptr, end=code.ptr+code.length;
// It is fine to skip decoding here, because we are just
// searching for ASCII characters.
// TODO: support unicode line breaks?
foreach_reverse(ref c; before)
if(c=='\n'||c=='\r'){start = c+1; break;}
foreach(ref c; after)
if(c=='\n'||c=='\r'){end = c; break;}
return start[0..end-start];
}
// ...
}
struct Location{
string rep;// slice of the code representing the Location
int line; // line number at start of location
@property Source source()const{
auto src = Source.get(rep); // (currently just a linear search)
assert(src, source for '~rep~' not found!);
return src;
}
// ...
}
int getColumn(Location loc, int tabsize){
int res=1;
auto l=loc.source.getLineOf(loc.rep);
for(;!l.emptyl[0]l.ptrloc.rep.ptr; l.popFront()){
if(l.front=='\t') res=res-res%tabsize+tabsize;
else res++;
}
return res;
}
Re: std.allocator needs your help
On 09/23/2013 09:11 PM, Andrei Alexandrescu wrote: On 9/23/13 11:55 AM, Robert Schadek wrote: On 09/23/2013 07:16 PM, Andrei Alexandrescu wrote: On 9/23/13 10:01 AM, deadalnix wrote: Finally, we got to decide how these basics block are used to form typed allocators, and interact with language constructs. Sure. Again: I describe the Otto cycle and you discuss how to paint the road. IMO there is a problem with this metaphor. If the road is painted metallic the Otto motor (cycle) needs cooling, otherwise ... boom. So I think the overall picture has to be kept in sight. Which makes the metaphor unintentionally better. Yes, we do need to worry about higher concerns because that's how layerd abstractions work. Andrei You're right! And I'm to tired to read properly.
Re: Debugging support for D - wiki
What about Win64 and VS Debugger?
Debugging support for D - wiki
I'm looking to begin adding integrated debugger support for the DDT IDE pretty soon. With this in mind it would be desirable to have a view of what level of D language debugger support is there for the various combinations of platform+compiler+debugger. This information would be quite beneficial to regular D users as well, as Manu's recent thread on the importance of a debugger is any indication of. Yet there doesn't seem to be any info about this in the wiki. The debuggers wiki page ( http://wiki.dlang.org/Debuggers ) doesn't even list the main players in this scene (VisualD/Mago, GDB, WinDebugger?) I might get started with this, but I would need to enlist the help of other people for the other platforms/debuggers I don't have proper acess to. The only combinations I tried so far was DMD+Windows+GDB, which seems like it's not supported at all. And GDC+Windows32+GDB which does seem to be well supported (GDB understands D name mangling, breakpoints in source, D data structures layout, etc.). I'm guessing GDC+GDB on Linux works just as well. (what about Mac though?) Several questions remain: Is debugger support with DMD+Linux+GDB as good as it is with GDC? For DMD+Windows, is there only good debugger support with VisualD? :-( And how well does that work with 32/64 bit platform variations? It would be great to have these answered, and going forward, keeping track of this info in the wiki. -- Bruno Medeiros - Software Engineer
Re: Move VisualD to github/d-programming-language ?
On 18/09/2013 14:41, eles wrote: On Wednesday, 18 September 2013 at 13:21:45 UTC, Bruno Medeiros wrote: On 18/09/2013 07:58, eles wrote: Not yet. How could I help? Actually, check my thread Debugging support for D - wiki. You might be able to help with research and trying out debuggers for configurations I can't (or simply haven't) tried myself. -- Bruno Medeiros - Software Engineer
Re: std.allocator needs your help
On 9/23/2013 12:09 PM, Andrei Alexandrescu wrote: I think there's at least indirect support in e.g. generating typeinfo objects appropriately. Many of the components of those typeinfos are expressly defined for helping a tracing collector. Those are not language features. Also, the latest support for that is done with a library-defined template.
Re: std.allocator needs your help
23-Sep-2013 22:52, Jacob Carlborg пишет: On 2013-09-23 17:47, Manu wrote: Imagine if new was a template (it would have to be a template). Allocating a template would require nested template syntax every time, which is pretty ugly. X x = new!(X!arg(args...)); // ewoo, paren spam... Why not: X x = new!X(args); +1 Or call it make for the time being so as to not collide with the keyword. -- Dmitry Olshansky
Re: Download page needs a tidy up
On 22/08/2013 04:23, Manu wrote: I got the latest GDC from dlang.org/download http://dlang.org/download, but it doesn't work. libiconv-2.dll (and possibly others?) is not included in the package. If this depends on another package being present, there is no text anywhere along the download path to inform the user. I just had the same problem today. I opened a thread in the GDC newsgroup to see if this can be rectified ( news://news.digitalmars.com:119/l1q33o$2m6i$1...@digitalmars.com ) as even on the GDC website there is no such info. But regardless, http://dlang.org/download should be updated as well. I agree on the point that is quite beneficial to D to have this information tidied up and better presented for new (and existing) D users. (even I took a while to figure out that I needed TDM-GCC in addition to the GDC packages in https://bitbucket.org/goshawk/gdc/downloads - and it's an installation procedure I've done in the past as well, but had forgotten). -- Bruno Medeiros - Software Engineer
Re: std.allocator needs your help
On 9/23/2013 9:47 AM, Benjamin Thaut wrote: I always understood void[] as block of unkown data. Untyped data, not unknown data.
Re: std.allocator needs your help
On 9/23/2013 10:08 AM, Andrei Alexandrescu wrote: I'm unclear on what void[] means starting from its semantics. I agree that it should be void[], as that represents (in D) a block of untyped data. void[] should be ideal for a primitive allocator.
Re: Download page needs a tidy up
On 22/08/2013 14:13, Ludovit Lucenic wrote: On Thursday, 22 August 2013 at 03:23:57 UTC, Manu wrote: I got the latest GDC from dlang.org/download, but it doesn't work. libiconv-2.dll (and possibly others?) is not included in the package. If this depends on another package being present, there is no text anywhere along the download path to inform the user. On 22 August 2013 13:07, Manu turkey...@gmail.com wrote: So I'm trying to find windows binaries for GDC and LDC... First place I look is dlang.org/download. Appears to be for DMD... keep looking. I look at the GDC/LDC wiki pages. No links to binaries anywhere. GDC and LDC home pages... no links to binaries. Github doesn't host binaries anymore... Where are they? Turns out there are links to the GDC binaries (hosted on bitbucket) on dlang.org/download. ...I didn't previously notice they were there, never scrolled down far enough. The impression you get from the top of the page is that dlang.orgis just DMD related, and I quickly dismissed it previously _ But there's still no LDC binary there... where is it? This needs to be fixed. You can argue I'm retarded and ignorant, but as an end user, it should take me no more than 5 seconds to find the download button. I suggest, on the front page of dlang.org, there should be a MASSIVE button: DOWNLOAD D COMPILERS, and the download page should be tweaked to be more obviously compiler agnostic. D1 and DMC consume an unreasonable amount of realestate, hiding GDC/LDC (surely basically nobody is looking for those?), perhaps they should be reduced to small test links with the other links down the bottom of the page? This will allow room to present GDC and LDC without scrolling. And why is there no LDC binary? I am completely with you, Manu. Same with me with GDC under Windows. Missing libiconv-2.dll... The same happened to me when looking around for the other two D compilers. It took me a couple of days until I found the link on the dlang.org's download page. Any suggestions on where to get a working copy of GDC to run on Windows, please? Thanks, Ludovit Check out: http://forum.dlang.org/thread/l1q33o$2m6i$1...@digitalmars.com I haven't tried the 64 bit alternative though (which has a more recent DMD frontend version: 2.060) -- Bruno Medeiros - Software Engineer
Re: std.allocator needs your help
On 9/23/13 1:04 PM, Walter Bright wrote: On 9/23/2013 10:08 AM, Andrei Alexandrescu wrote: I'm unclear on what void[] means starting from its semantics. I agree that it should be void[], as that represents (in D) a block of untyped data. void[] should be ideal for a primitive allocator. Great, made that change, it all works. Does void[] make any promise about alignment? Andrei
Re: std.allocator needs your help
On 09/23/2013 01:49 AM, Andrei Alexandrescu wrote:
I am making good progress on the design of std.allocator, and I am
optimistic about the way it turns out. D's introspection capabilities
really shine through, and in places the design does feel really
archetypal - e.g. this is the essence of a freelist allocator. It's a
very good feeling. The overall inspiration comes from Berger's
HeapLayers, but D's introspection takes that pattern to a whole new level.
...
Seems neat.
Several details are still in flux, but at the top level it seems most
natural to divide things in two tiers:
1. Typed allocator, i.e. every request for allocation comes with the
exact type requested;
2. Untyped allocator - traffics exclusively in ubyte[].
...
Some general remarks:
One issue you will probably run into and maybe want to fix in some way
during the typed allocator design is that private constructors cannot be
called from templates parameterized on types.
E.g.:
module a;
auto New(T,Args...)(Args args){
return new T(args);
}
// ---
module b;
class A{
private this(){}
}
void main(){
auto a = New!A(); // error
}
Is there an intention to also support replacements of instance new
allocations, i.e. object.new subobject(args)?
Re: std.allocator needs your help
On Sunday, 22 September 2013 at 23:49:56 UTC, Andrei Alexandrescu wrote: 1. Typed allocator, i.e. every request for allocation comes with the exact type requested; 2. Untyped allocator - traffics exclusively in ubyte[]. This is a good design. Perhaps this is a later concern, but we should make sure that node-based containers (e.g. linked list, trees) have a way to access the allocation size needed for the node. STL did not do this. * alignment is a compile-time constant yielding the alignment of allocated data. Many allocators offer the maximum alignment of the platform (for which I used real.alignof above). This constant is required for all allocators. What if, for example, I wanted to allocate a 4096 byte aligned block from the GC allocator? Do I have to create a new allocator backed by the GC allocator? What if the alignment is not known at compile time (e.g. hard disk page size or CPU cache line size)? Might be better to pass the desired alignment in the allocate method. * available is a property that returns how many total (not necessarily contiguous) bytes are available for allocation. The NullAllocator knows statically it has 0 bytes available so it implements it as an enum. Generally allocators will implement it as a @property. This property is optional. It would be useful to know the maximum available contiguous block size too, so that you can find out if an allocation will succeed without calling allocate. It's also useful for diagnosing fragmentation issues e.g. allocation failed, free memory = X, max contiguous = Y. If X is high and Y is low then you are highly fragmented. Of course, this should be optional. * allocate(s) returns a ubyte[] with length at least s, or null. (It does not throw on out-of-memory because that would hurt composability; it turns out many elemental allocators do naturally run out of memory.) This method is required for all allocators. In most allocators this method should be @safe. What are the semantics of allocate(0)? malloc(0) is implementation defined.
Re: std.allocator needs your help
On 9/23/13 1:32 PM, Timon Gehr wrote: One issue you will probably run into and maybe want to fix in some way during the typed allocator design is that private constructors cannot be called from templates parameterized on types. Hrm, that's a real problem. We've ran into it at work when we were trying to replace a macro with a template. The macro is automatically friend with everyone! :o) Is there an intention to also support replacements of instance new allocations, i.e. object.new subobject(args)? It's a good point to keep in mind as we move further with this. Andrei
Re: [OT] Which IDE / Editor do you use?
On Mon, 23 Sep 2013 07:50:43 -0700 H. S. Teoh hst...@quickfur.ath.cx wrote: On Mon, Sep 23, 2013 at 02:01:02PM +0200, Dicebot wrote: Ironically, this is exactly the reason I have never succeeded in using the Windows for daily work. Amount of manual configuration and subverting the defaults needed to make it actually usable for my programming flow is outstanding. In the same time on my Linux distro it is mostly `pacman -Sy gnome gnome-extra xorg-server nvidia dlang vim git` and I am ready to work on a fresh install. Ditto. Similar here, at least these days. Last time I got a new system, it had Win7 preinstalled. I'd already been using Windows as my primary system (for ages), but it took me about *a week or two* of googling and tweaking and third party hacks before I had finally got it to a point that was more or less usable (but still frequently irritating and impractical). I'm surprised at people talking about the amount of time spent configuring stuff on Linux, etc., because it's never happened to me! I mean, OK, in the early days you had to manually configure X11 and deal with all of the obscure problems, but that's no longer the case today. All I have to do is 'apt-get install package' and it Just Works(tm). Yea, I find installing software is often (not always, but often) *easier* on Linux these days, thanks to apt and such. It used to be a nightmare of dependency issues in the days of dpkg/rpm[1]. But now, most of the time it's just one trivial command. Done. Or if you prefer GUIs, the...uhh...what's it called, Synaptec?, is just as easy, *and* good for discoverability: It's basically a freeware app store, but Linux had it even *before* iOS. And when you install a GUI app it puts *one* item in the start menu, not Company Name - Program Name - Tons of Junk including docs and an uninstall link that's right next to the launch link Of course, sometimes windows is still easier, too. I get so frustrated when I discover some Posix program through the web and it gives *no* indication how to download or install it because they just assume it's already in everyone's distro repos and that everyone knows what their distro named the package, and that you're installing it *on* the same Posix distro you're already familiar with. For example, last night I tried to install the cmdline 7z on an OSX machine, and after nearly an hour of completely useless *official* webpages, and screwing around with sources, etc, I gave up. [1] Of course I know dpkg/rpm are still there under the hood. I do like to tweak stuff -- and this is where Windows falls flat for me, 'cos it forces you to work a certain way, and when you go outside of that, things just stop working or it becomes an uphill battle (disclaimer: I haven't touched Windows for over a decade, so this may no longer be true today) That's actually *more* true now than it was with XP. MS used to be pretty big on preferences, customization, etc (maybe not *AS* much as Linux, but still). But lately they've been very noticeably trying to become Apple/Mozilla, complete with forcing every change on everybody and minimizing how much they'll allow you to revert any of their endless stream of crazy UI flavor of the month ideas.
Re: std.allocator needs your help
On 9/23/13 1:37 PM, Peter Alexander wrote:
What if, for example, I wanted to allocate a 4096 byte aligned block
from the GC allocator? Do I have to create a new allocator backed by the
GC allocator?
I've been thinking a good library component would be AlignmentAllocator
that would work like this:
struct AlignmentAllocator(Allocator, size_t desiredAlignment)
if (Allocator.alignment desiredAlignment)
{
enum alignment = desiredAlignment;
...
}
That allocator would allocate more memory (I suspect there's a gcd
calculation somewhere :o)) and then adjust the starting pointer of the
allocated block to reach the requested alignment.
I'm just off the phone with Walter discussing this and related issue.
It's an interesting problem space.
What if the alignment is not known at compile time (e.g. hard disk page
size or CPU cache line size)?
The proposed design assumes compile-time allocation sizes. A trick
similar to the one above (overallocate and adjust pointer) should work.
I'd need a handful of good examples where the alignment must be known at
runtime. Can you link to some?
Might be better to pass the desired alignment in the allocate method.
The thing is in 99%+ of the cases you don't need it. Then perhaps in
99%+ of the remaining cases the alignment is known during compilation.
Nevertheless it's a change worth investigating.
* available is a property that returns how many total (not necessarily
contiguous) bytes are available for allocation. The NullAllocator
knows statically it has 0 bytes available so it implements it as an
enum. Generally allocators will implement it as a @property. This
property is optional.
It would be useful to know the maximum available contiguous block size
too, so that you can find out if an allocation will succeed without
calling allocate.
I think the best way is to just go ahead and attempt an allocation,
especially if the allocator is shared (races!). (There's some
flexibility with expand() there, which has a minimum size and a desired
size.) But clearly the information of the largest contiguous block is
currently missing, and is reasonable to be considered for addition.
It's also useful for diagnosing fragmentation issues
e.g. allocation failed, free memory = X, max contiguous = Y. If X is
high and Y is low then you are highly fragmented.
Of course, this should be optional.
Makes sense.
* allocate(s) returns a ubyte[] with length at least s, or null. (It
does not throw on out-of-memory because that would hurt composability;
it turns out many elemental allocators do naturally run out of
memory.) This method is required for all allocators. In most
allocators this method should be @safe.
What are the semantics of allocate(0)? malloc(0) is implementation defined.
I defined Mallocator to return whatever malloc returns on allocate(0),
but decided realloc() is too messed up and special-cased reallocate(0)
to free memory and place null in the buffer. Probably I'll work the same
logic in Mallocator.allocate(0) to eliminate platform dependencies.
Andrei
Re: std.allocator needs your help
On 9/23/2013 1:18 PM, Andrei Alexandrescu wrote: Does void[] make any promise about alignment? No.
