[Issue 18140] experimental allocators, bitmapped_block UB on OSX 64 bits

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18140

RazvanN  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||razvan.nitu1...@gmail.com
 Resolution|--- |WORKSFORME

--- Comment #3 from RazvanN  ---
This seems to have been fixed, the autotester is green.

--

[Issue 23090] Allocators should not use NullAllocator as a sentinel type

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23090

Iain Buclaw  changed:

   What|Removed |Added

   Priority|P1  |P4

--

[Issue 23090] Allocators should not use NullAllocator as a sentinel type

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23090

--- Comment #1 from Dlang Bot  ---
@pbackus updated dlang/phobos pull request #8574 "[allocator/region] Issue
23090 - Don't use NullAllocator as a sentinel type" mentioning this issue:

- Region: don't use NullAllocator as a sentinel

  Region will now always assume that its ParentAllocator is a real
  allocator, and attempt to use its 'allocate' and 'deallocate' methods
  accordingly.

  The behavior previously provided by Region!(NullAllocator, ...) remains
  available via BorrowedRegion!(...).

  See issue 23090 for detailed rationale.

- SharedRegion: don't use NullAllocator as sentinel

  SharedRegion will now always assume that its ParentAllocator is a real
  allocator, and attempt to use its 'allocate' and 'deallocate' methods
  accordingly.

  The behavior previously provided by SharedRegion!(NullAllocator, ...)
  remains available via shared(BorrowedRegion!(...)).

  See issue 23090 for detailed rationale.

https://github.com/dlang/phobos/pull/8574

--

[Issue 23090] New: Allocators should not use NullAllocator as a sentinel type

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=23090

  Issue ID: 23090
   Summary: Allocators should not use NullAllocator as a sentinel
type
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: snarwin+bugzi...@gmail.com

Several allocators in std.experimental.allocator have special-case behavior
when instantiated using the specific type `NullAllocator`.

A non-exhaustive list:

* Region: "If ParentAllocator is different from NullAllocator, Region
deallocates the chunk of memory during destruction." [1]

* ContiguousFreeList: "The block is assumed to have been allocated with
ParentAllocator, and is released in ContiguousFreeList's destructor (unless
ParentAllocator is NullAllocator)." [2]

* AllocatorList: "If BookkeepingAllocator is NullAllocator, then AllocatorList
is "ouroboros-style", i.e. it keeps the bookkeeping data in memory obtained
from the allocators themselves." [3]

It is clear from these examples that the type `NullAllocator` is being used as
a sentinel to indicate the *absence* of an allocator, rather than for its
actual documented purpose of being an allocator that is always out of memory.

This abuse of NullAllocator has two undesirable consequences:

1. Users who want NullAllocator's out-of-memory behavior must define their own
version with a different name, rather than using NullAllocator directly.
2. Readers of the std.experimental.allocator documentation are likely to come
away confused about NullAllocator's purpose.

[1]
https://dlang.org/library/std/experimental/allocator/building_blocks/free_list/contiguous_free_list.html
[2]
https://dlang.org/library/std/experimental/allocator/building_blocks/free_list/contiguous_free_list.html
[3]
https://dlang.org/library/std/experimental/allocator/building_blocks/allocator_list/allocator_list.html

--

Re: Are there any containers that go with allocators?

[ryuukk_ via Digitalmars-d-learn]

On Tuesday, 9 February 2021 at 12:18:41 UTC, John Burton wrote:
Normally I'm happy with the GC containers in D, they work well 
and suit my use.


I have a few uses that would benefit from allocation in memory 
arenas or local stack based allocation. Looks like 
std.experimental has allocators for those use cases.


But I can't find any containers to make use of those 
allocators. The built in ones use GC managed memory, there are 
std.container but they appear to use malloc/free which again is 
useful but not what I'm looking for.


I'd like resizeable arrays, hashmaps and strings that I can 
allocate using an allocator?
Am I missing something in the standard library, or a way to use 
the existing ones, or is it just that nobody has implemented 
this kind of thing yet?


i use https://github.com/ikod/ikod-containers

Re: Are there any containers that go with allocators?

[John Burton via Digitalmars-d-learn]

On Tuesday, 9 February 2021 at 12:23:52 UTC, rikki cattermole 
wrote:

https://github.com/dlang-community/containers

It uses the older design for allocators (dependency).


Looks good, thank you

Re: Are there any containers that go with allocators?

[rikki cattermole via Digitalmars-d-learn]

https://github.com/dlang-community/containers

It uses the older design for allocators (dependency).

Are there any containers that go with allocators?

[John Burton via Digitalmars-d-learn]

Normally I'm happy with the GC containers in D, they work well 
and suit my use.


I have a few uses that would benefit from allocation in memory 
arenas or local stack based allocation. Looks like 
std.experimental has allocators for those use cases.


But I can't find any containers to make use of those allocators. 
The built in ones use GC managed memory, there are std.container 
but they appear to use malloc/free which again is useful but not 
what I'm looking for.


I'd like resizeable arrays, hashmaps and strings that I can 
allocate using an allocator?
Am I missing something in the standard library, or a way to use 
the existing ones, or is it just that nobody has implemented this 
kind of thing yet?

Re: Class Allocators

[evilrat via Digitalmars-d-learn]

On Sunday, 31 January 2021 at 23:19:09 UTC, Kyle wrote:
My best guess right now is that both class allocators and the 
placement new syntax are deprecated, but if that's the case I 
would expect a deprecation message when I try to use that 
new(address) Type syntax whether there's a class allocator 
present or not. Any insight into this? Thanks.


Yes, just use emplace() insead of placement new.

GC-less allocations however is up to you, either malloc/free, 
std.experimental.allocator or any other way. You can make your 
own smart pointer struct to handle this automatically, or better 
use community packages such as 'automem'.


As for the message it is possible that this part of the reference 
compiler was already passed deprecation period and should be 
removed but was completely forgotten.


https://dlang.org/phobos/core_lifetime.html#.emplace

Re: Class Allocators

[Mike Parker via Digitalmars-d-learn]

On Sunday, 31 January 2021 at 23:19:09 UTC, Kyle wrote:

strategy outside of the class". My best guess right now is that 
both class allocators and the placement new syntax are 
deprecated, but if that's the case I would expect a deprecation 
message when I try to use that new(address) Type syntax whether 
there's a class allocator present or not. Any insight into 
this? Thanks.


Class allocators were deprecated in 2.080 and became an error in 
2.087:


https://dlang.org/deprecate.html

I assume the placement new syntax was supposed to have been 
deprecated as well. It's possible it was overlooked, or that it 
wasn't deprecated because it has an obscure use-case that I'm 
unaware of. These days we should be using std.conv.emplace:


https://dlang.org/phobos/std_conv.html#.emplace.4

See also:

http://p0nce.github.io/d-idioms/#Placement-new-with-emplace

I suggest you file an issue at:

https://issues.dlang.org/

Class Allocators

[Kyle via Digitalmars-d-learn]

I've been doing some review of Andrei's book which has led me to 
trying to figure out how placement new works, or doesn't work. 
The new(address) Type syntax tells me "no allocator for TYPE". I 
did find information on "Class Allocators" at 
https://dlang.org/spec/class.html#allocators, but when I try to 
do things the Class Allocator way DMD tells me that "class 
allocators are obselete, consider moving the allocation strategy 
outside of the class". My best guess right now is that both class 
allocators and the placement new syntax are deprecated, but if 
that's the case I would expect a deprecation message when I try 
to use that new(address) Type syntax whether there's a class 
allocator present or not. Any insight into this? Thanks.

[Issue 12081] Warn against, and then deprecate class allocators

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=12081

Basile-z  changed:

   What|Removed |Added

 CC|b2.t...@gmx.com |

--

[Issue 16398] experimental allocators, add aligned reallocation for Posix

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16398

Basile-z  changed:

   What|Removed |Added

 CC|b2.t...@gmx.com |

--

[Issue 15197] Allow allocator_list to work with allocators that do not support `expand()`

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15197

berni44  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||bugzi...@d-ecke.de
 Resolution|--- |WORKSFORME

--

[Issue 12081] Warn against, and then deprecate class allocators

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=12081

Basile-z  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||b2.t...@gmx.com
 Resolution|--- |FIXED

--- Comment #1 from Basile-z  ---
They've been removed now. Only one limited form remains, with @disable, to
allow blocking the 'new' operator.

--

[Issue 18140] experimental allocators, bitmapped_block UB on OSX 64 bits

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18140

Basile B.  changed:

   What|Removed |Added

   Severity|normal  |blocker

--

[Issue 18140] experimental allocators, bitmapped_block UB on OSX 64 bits

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18140

Basile B.  changed:

   What|Removed |Added

 CC||bugzi...@digitalmars.com

--- Comment #2 from Basile B.  ---
*** Issue 18914 has been marked as a duplicate of this issue. ***

--

[Issue 18140] experimental allocators, bitmapped_block UB on OSX 64 bits

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18140

Basile B.  changed:

   What|Removed |Added

 CC||greensunn...@gmail.com

--- Comment #1 from Basile B.  ---
*** Issue 18262 has been marked as a duplicate of this issue. ***

--

[Issue 18140] New: experimental allocators, bitmapped_block UB on OSX 64 bits

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18140

  Issue ID: 18140
   Summary: experimental allocators, bitmapped_block UB on OSX 64
bits
   Product: D
   Version: D2
  Hardware: x86_64
OS: Mac OS X
Status: NEW
  Severity: normal
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: b2.t...@gmx.com

See
https://auto-tester.puremagic.com/show-run.ghtml?projectid=14=2968834=true.

** FAIL debug64 std.experimental.allocator.building_blocks.bitmapped_block
core.exception.AssertError@std/experimental/allocator/building_blocks/bitmapped_block.d(173):
Data must be aligned properly

4   test_runner 0x00011017985b onAssertErrorMsg +
107
5   test_runner 0x000110179ab8 _d_assert_msg + 68
6   test_runner 0x00010fd0d20f pure ref
std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock
std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock.__ctor(ubyte[])
+ 111
7   test_runner 0x00010fd0d3a3 pure ref
std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock
std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock.__ctor(ubyte[],
uint) + 51
8   test_runner 0x00010fd0ff3f pure
std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock
std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock.__unittest_std_experimental_allocator_building_blocks_bitmapped_block_d_53_17().__lambda2()
+ 103
9   test_runner 0x00010fd0f6f7 void
std.experimental.allocator.common.testAllocator!(std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock.__unittest_std_experimental_allocator_building_blocks_bitmapped_block_d_53_17().__lambda2()).testAllocator()
+ 39
10  test_runner 0x00010fd0d002 void
std.experimental.allocator.building_blocks.bitmapped_block.BitmappedBlock!(-2uL,
64u,
std.experimental.allocator.building_blocks.null_allocator.NullAllocator).BitmappedBlock.__unittest_std_experimental_allocator_building_blocks_bitmapped_block_d_53_17()
+ 74
11  test_runner 0x00010fd04cfb void
std.experimental.allocator.building_blocks.bitmapped_block.__modtest() + 103
12  test_runner 0x00010ecaf7d3 void
test_runner.doTest(object.ModuleInfo*, ref core.runtime.UnitTestResult) + 99
13  test_runner 0x00010ecaf6db
core.runtime.UnitTestResult test_runner.testModules() + 395
14  test_runner 0x00010ecaf527
core.runtime.UnitTestResult test_runner.tester() + 31
15  test_runner 0x00011017a593 runModuleUnitTests +
151
16  test_runner 0x00011018f2a5 void
rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll()
+ 37
17  test_runner 0x00011018f223 void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate()) + 51
18  test_runner 0x00011018f186 _d_run_main + 814
19  test_runner 0x00010ecaf3d5 main + 33
20  test_runner 0x00010ecaf393 start + 51
21  ??? 0x0001 0x0 + 1

--

[Issue 16398] experimental allocators, add aligned reallocation for Posix

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16398

--- Comment #4 from github-bugzi...@puremagic.com ---
Commits pushed to stable at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/a196f171f6cc0d6936d7b2d5fc4fc7e38b026514
fix issue 16398 - experimental allocators, add aligned reallocation for Posix

https://github.com/dlang/phobos/commit/0308bd6b9a080501b931f43d1e39b1c3cf79c29a
Merge pull request #5857 from BBasile/issue-16398

--

[Issue 16398] experimental allocators, add aligned reallocation for Posix

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16398

--- Comment #3 from github-bugzi...@puremagic.com ---
Commits pushed to master at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/a196f171f6cc0d6936d7b2d5fc4fc7e38b026514
fix issue 16398 - experimental allocators, add aligned reallocation for Posix

https://github.com/dlang/phobos/commit/0308bd6b9a080501b931f43d1e39b1c3cf79c29a
Merge pull request #5857 from BBasile/issue-16398

fix issue 16398 - experimental allocators, add aligned reallocation for Posix

--

[Issue 16398] experimental allocators, add aligned reallocation for Posix

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16398

Basile B.  changed:

   What|Removed |Added

   Keywords||pull

--- Comment #2 from Basile B.  ---
PR https://github.com/dlang/phobos/pull/5857

--

[Issue 16398] experimental allocators, add aligned reallocation for Posix

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=16398

Basile B. <b2.t...@gmx.com> changed:

   What|Removed |Added

Summary|aligned allocator: add  |experimental allocators,
   |aligned reallocation for|add aligned reallocation
   |Posix   |for Posix

--

Re: Class allocators

[Nordlöw via Digitalmars-d-learn]

On Sunday, 12 November 2017 at 20:41:03 UTC, Basile B. wrote:
No, the classes and structs of the examples are simply declared 
as 'static' because they are located in a 'unittest' block. You 
can ignore the keyword...it just means that they are declared 
as if they would stand at the global scope.


Thanks.

Re: Class allocators

[Basile B. via Digitalmars-d-learn]

On Sunday, 12 November 2017 at 18:46:54 UTC, Per Nordlöw wrote:
On Sunday, 12 November 2017 at 18:34:42 UTC, Eduard Staniloiu 
wrote:

On Saturday, 11 November 2017 at 14:26:34 UTC, Nordlöw wrote:

Have anybody used allocators to construct class instances?


I might be wrong, but I think you are looking for 
std.experimental.allocator.make [0]


[0] - 
https://dlang.org/phobos/std_experimental_allocator.html#make


Thanks!

In the example the classes are declared as static. What effect 
does this have here and is it required for classes allocated 
via make?


No, the classes and structs of the examples are simply declared 
as 'static' because they are located in a 'unittest' block. You 
can ignore the keyword...it just means that they are declared as 
if they would stand at the global scope.

Re: Class allocators

[Per Nordlöw via Digitalmars-d-learn]

On Sunday, 12 November 2017 at 18:34:42 UTC, Eduard Staniloiu 
wrote:

On Saturday, 11 November 2017 at 14:26:34 UTC, Nordlöw wrote:

Have anybody used allocators to construct class instances?


I might be wrong, but I think you are looking for 
std.experimental.allocator.make [0]


[0] - 
https://dlang.org/phobos/std_experimental_allocator.html#make


Thanks!

In the example the classes are declared as static. What effect 
does this have here and is it required for classes allocated via 
make?

Re: Class allocators

[Eduard Staniloiu via Digitalmars-d-learn]

On Saturday, 11 November 2017 at 14:26:34 UTC, Nordlöw wrote:

Have anybody used allocators to construct class instances?


I might be wrong, but I think you are looking for 
std.experimental.allocator.make [0]


[0] - 
https://dlang.org/phobos/std_experimental_allocator.html#make

Re: Class allocators

[Eugene Wissner via Digitalmars-d-learn]

On Saturday, 11 November 2017 at 14:26:34 UTC, Nordlöw wrote:

Have anybody used allocators to construct class instances?


Do you mean phobos allocators? or allocators as concept?
What is the problem?

Class allocators

[Nordlöw via Digitalmars-d-learn]

Have anybody used allocators to construct class instances?

Re: Containers and arrays with custom memory allocators

[Per Nordlöw via Digitalmars-d-learn]

On Thursday, 19 October 2017 at 08:47:09 UTC, Per Nordlöw wrote:
These are all very performant containers, but currently lack 
support for std.experimental.allocator.


Support for std.experimental.allocator is planned but currently 
not a priority. EMSI-containers have an elegant integration and 
will be an inspiration in the process.

Re: Containers and arrays with custom memory allocators

[Per Nordlöw via Digitalmars-d-learn]

On Thursday, 19 October 2017 at 08:47:09 UTC, Per Nordlöw wrote:

explicit via .dup member and when you want to copy or pass by


should be:

explicit via .dup member and when you want to _move_ from one 
l-value to another l-value or pass by move

Re: Containers and arrays with custom memory allocators

[Per Nordlöw via Digitalmars-d-learn]

On Thursday, 19 October 2017 at 08:47:09 UTC, Per Nordlöw wrote:
The file hashmap.d provides both a HashSet and HashMap 
implementation in one go using D's 
compile-time-code-branching-mechanism `static if`.


Correction:

I've now broken it up into

- 
https://github.com/nordlow/phobos-next/blob/master/src/hashmap_or_hashset.d

- https://github.com/nordlow/phobos-next/blob/master/src/hashset.d
- https://github.com/nordlow/phobos-next/blob/master/src/hashmap.d

Re: Containers and arrays with custom memory allocators

[Per Nordlöw via Digitalmars-d-learn]

On Tuesday, 17 October 2017 at 14:14:19 UTC, Ivan wrote:

Hi,

I am a C/C++ programmer interested in using D as a replacement 
for C/C++.


I do care a lot about performance and memory management, so
I want to use my own (or from std.experimental) memory
allocators.

Are there any good tutorials or examples about how to use
custom memory allocators for arrays and existing containers?

Or should I just go ahead and write my own containers that are 
allocator

aware?

Thanks.


I'm currently working on a set of containers as part of my very 
general repo


https://github.com/nordlow/phobos-next/tree/master/src

For instance:

- 
https://github.com/nordlow/phobos-next/blob/master/src/basic_array.d
- 
https://github.com/nordlow/phobos-next/blob/master/src/bitarray.d
- 
https://github.com/nordlow/phobos-next/blob/master/src/static_bitarray.d
- 
https://github.com/nordlow/phobos-next/blob/master/src/fixed_array.d

- https://github.com/nordlow/phobos-next/blob/master/src/soa.d
- https://github.com/nordlow/phobos-next/blob/master/src/hashmap.d

and a lightweight polymorphic container at

- 
https://github.com/nordlow/phobos-next/blob/master/src/variant_arrays.d


and a very experimental radix-tree (trie) implementation at

- https://github.com/nordlow/phobos-next/blob/master/src/trie.d

These are all very performant containers, but currently lack 
support for std.experimental.allocator. Instead they use C-style 
allocation (via qualified C-memory management in qcmeman.c). 
Further they use Rust-like semantics via disabling of 
copy-construction; `@disable this(this);`. Instead copying (like 
for EMSI-containers and many others) is forced to be explicit via 
.dup member and when you want to copy or pass by move use, for 
instance,


import std.algorithm.mutation : move;
import basic_array : A = BasicArray;

Ai = A!int;
auto s = Ai.withLength(10);

Ai src, dst;
move(src, dst);

someFunctionTakingArrayByValue(move(s)); // pass by move

The file hashmap.d provides both a HashSet and HashMap 
implementation in one go using D's 
compile-time-code-branching-mechanism `static if`.


If you want reference semantics (via reference counting) each 
container can be wrapper in a `std.typecons.RefCounted` for 
instance as


import basic_array : A = BasicArray;
alias Ai = A!int;
import std.typecons : RefCounted;
RefCounted!A x;

used here

https://github.com/nordlow/phobos-next/blob/master/src/basic_array.d#L1288

Further note that my work has focused heavily on making things 
`@safe pure nothrow @nogc` via DIP-25/DIP-1000 when possible 
which is not the case with most other D container libraries I've 
tried. In some cases I might have made a mistake with my @trusted 
taggings. Please correct me if that is the case. Also note that 
DIP-1000 scope analysis doesn't currently kick in correctly in 
templated containers because of a bug in the compiler. The bug 
has been filed at bugzilla and my guess is that it will soon be 
fixed, as making scope analysis more complete is a high priority, 
at least for Walter Bright.


~master currently builds with both dmd (debug mode only) and ldc2 
(both debug and release mode). I'm currently searching for some 
part of trie.d that currently makes dmd segfault when compiled in 
release mode with inlining enabled. I think DMD's inlining is the 
root of the problem so be careful when using trie.d.


My preliminary benchmark at

https://github.com/nordlow/phobos-next/blob/master/benchmarks/source/app.d

compiled with LDC and inlining enabled shows that 
`HashMap!(uint/ulong, uint/ulong)`'s `insert()` and `contains()` 
with FNV64 hash is at least 10 times as fast as D's builtin 
associative arrays on my Intel Haswell laptop.

Re: Containers and arrays with custom memory allocators

[Fra Mecca via Digitalmars-d-learn]

On Tuesday, 17 October 2017 at 14:14:19 UTC, Ivan wrote:

Hi,

I am a C/C++ programmer interested in using D as a replacement 
for C/C++.


I do care a lot about performance and memory management, so
I want to use my own (or from std.experimental) memory
allocators.

Are there any good tutorials or examples about how to use
custom memory allocators for arrays and existing containers?

Or should I just go ahead and write my own containers that are 
allocator

aware?

Thanks.


I am still working on it, but given your case it may be useful.
https://github.com/FraMecca/D_Libraries_Registry

Re: Containers and arrays with custom memory allocators

[Aldo via Digitalmars-d-learn]

On Tuesday, 17 October 2017 at 14:14:19 UTC, Ivan wrote:

Hi,

I am a C/C++ programmer interested in using D as a replacement 
for C/C++.


I do care a lot about performance and memory management, so
I want to use my own (or from std.experimental) memory
allocators.

Are there any good tutorials or examples about how to use
custom memory allocators for arrays and existing containers?

Or should I just go ahead and write my own containers that are 
allocator

aware?

Thanks.


You can check this repo :

https://github.com/economicmodeling/containers

It contains containers backed by std.experimental.allocator.

Containers and arrays with custom memory allocators

[Ivan via Digitalmars-d-learn]

Hi,

I am a C/C++ programmer interested in using D as a replacement 
for C/C++.


I do care a lot about performance and memory management, so
I want to use my own (or from std.experimental) memory
allocators.

Are there any good tutorials or examples about how to use
custom memory allocators for arrays and existing containers?

Or should I just go ahead and write my own containers that are 
allocator

aware?

Thanks.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Sunday, 7 May 2017 at 11:46:33 UTC, Andrei Alexandrescu wrote:

On 5/7/17 1:30 PM, Stanislav Blinov wrote:




To follow this discussion up, and in light of yesterday's 
Collections presentation at DConf, I though I'd go ahead and 
make a basic implementation of traits-based IAllocator.


You can find it here:

https://github.com/radcapricorn/alloctraits


Cool, thanks. Before we saw that, Eduard and I are working on a 
similar solution along the lines of IAllocator!(Flag!"nogc", 
Flag!"safe"). -- Andrei


I guess the exact representation of "traits" is something to 
decide on. I have to say, even though I used the BitFlags-based 
solution, I'm not a big fan of that, the syntax is a bit 
unwieldy. Then again, Flag-based would be too.

Perhaps a struct with enum members could be used...

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Andrei Alexandrescu via Digitalmars-d]

On 5/7/17 1:30 PM, Stanislav Blinov wrote:




To follow this discussion up, and in light of yesterday's Collections 
presentation at DConf, I though I'd go ahead and make a basic 
implementation of traits-based IAllocator.


You can find it here:

https://github.com/radcapricorn/alloctraits


Cool, thanks. Before we saw that, Eduard and I are working on a similar 
solution along the lines of IAllocator!(Flag!"nogc", Flag!"safe"). -- Andrei

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Monday, 1 May 2017 at 13:00:27 UTC, Andrei Alexandrescu wrote:

On 05/01/2017 08:12 AM, Guillaume Piolat wrote:
On Sunday, 30 April 2017 at 21:43:26 UTC, Andrei Alexandrescu 
wrote:


A pass through the root allocators (Mallocator, GCAllocator 
etc)
figuring out what attributes could be meaningfully attached 
would be

welcome. The rest would rely on inference.


Thanks,

Andrei


IAllocator being fully @nogc would be a comforting guarantee, 
as runtime

dispatch makes for lighter types.


As I said (and am encouraging you to do this), this is achieved 
through simple variance:


interface INoGCAllocator : IAllocator {
   ... override all methods here as @nogc ...
}

This is possible because a @nogc method may override one that 
is not @nogc. @nogc being more restrictive it is contravariant 
with no-@nogc.


Also IAllocator should have a few @nogc methods to start with; 
there's no reason e.g. for empty() to create garbage.


Could you please initiate a PR?


Andrei


To follow this discussion up, and in light of yesterday's 
Collections presentation at DConf, I though I'd go ahead and make 
a basic implementation of traits-based IAllocator.


You can find it here:

https://github.com/radcapricorn/alloctraits

It pretty much evades the shortcomings of IAllocator, alleviates 
the need for any additional interfaces (ISharedAllocator et al.), 
at the cost of making user code slightly more verbose. But that 
is a small price to pay if we want to benefit from the type 
system.
The current implementation lacks a few features of 
std.experimental.allocator (i.e. passing implementation by 
pointer), which are easy to add, I was just focused mainly on 
roughing out the design as the first step.


Anyone interested, feel free to comment, improve, bash, destroy...

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Atila Neves via Digitalmars-d]

On Monday, 1 May 2017 at 07:42:19 UTC, Sebastiaan Koppe wrote:

On Friday, 28 April 2017 at 22:18:54 UTC, Atila Neves wrote:
Done. I also added to the README that it has its own versions 
of the range constraints from Phobos that can be used with 
`@models`.


Atila


Example of an error message in the README would be great too.


Good point, done.

Atila

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 05/01/2017 10:51 PM, Stanislav Blinov wrote:

On Monday, 1 May 2017 at 16:31:10 UTC, Nick Sabalausky (Abscissa) wrote:


If we had a type similar to TaggedAlgebraic...

Destroy?


It's too strict: you have to specify concrete types beforehand.


No, that's why my suggestion was:

"If we had a type similar to TaggedAlgebraic, but was an open variant 
rather than a closed algebraic (Ie, a type like Phobos's Variant, but 
forwarded member access without first requiring explicit conversion to 
the exact underlying type),"

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Tuesday, 2 May 2017 at 02:51:02 UTC, Stanislav Blinov wrote:

Lost one else. Should be

 static if (traits & (AllocatorTraits.sharedInstance |
 AllocatorTraits.noGC))
 @nogc shared { mixin AllocatorInterface!(); }
 else static if (traits & AllocatorTraits.sharedInstance)
 shared { mixin AllocatorInterface!(); }
 else static if (traits & AllocatorTraits.noGC)
 @nogc { mixin AllocatorInterface!(); }
 else
 mixin AllocatorInterface!();

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Monday, 1 May 2017 at 16:31:10 UTC, Nick Sabalausky (Abscissa) 
wrote:



If we had a type similar to TaggedAlgebraic...

Destroy?


It's too strict: you have to specify concrete types beforehand. 
This spills over into user code and makes it far less versatile 
that can be achieved.
Currently with allocators we have two extremes: IAllocator (and 
now, ISharedAllocator) that carries *no* static type information 
about concrete implementation, *and* loses all the inference 
regarding @nogc and shared memory. On the other end it's 
CAllocatorImpl that carries *both*.
But for user code, what is desirable is to have a middle man: a 
type that erases concrete allocator type but still keeps the 
@nogc and/or shared. This would enable user types to exchange 
their allocators while keeping static checks afforded by the 
language.

This can be done e.g. like this:

enum AllocatorTraits
{
none= 0x00,
noGC= 0x01,
sharedMemory= 0x02,
sharedInstance  = 0x04,
}

mixin template AllocatorInterface()
{
Ternary owns(Block block) @nogc;
Ternary resolveInternalPointer(void* p, ref Block result) 
@nogc;

Ternary empty() const @nogc;

Block allocate(size_t, TypeInfo ti = null);
Block alignedAllocate(size_t, uint);
Block allocateAll();
bool expand(ref Block, size_t);
bool reallocate(ref Block, size_t);
bool alignedReallocate(ref Block, size_t, uint);
bool deallocate(Block block);
bool deallocateAll();
}

template IAllocatorImpl(AllocatorTraits traits)
{
static if (traits & AllocatorTraits.sharedInstance)
alias Base = ISharedAllocator;
else
alias Base = IAllocator;

interface IAllocatorImpl : Base
{
static if (traits & AllocatorTraits.sharedMemory)
alias Block = shared(void)[];
else
alias Block = void[];

static if (traits & (AllocatorTraits.sharedInstance | 
AllocatorTraits.noGC))

@nogc shared { mixin AllocatorInterface!(); }
else static if (traits & AllocatorTraits.sharedInstance)
shared { mixin AllocatorInterface!(); }
else
@nogc { mixin AllocatorInterface!(); }

mixin AllocatorInterface!();
}
}

...and make the allocatorObject() function return 
IAllocatorImpl!traits instead of CAllocatorImpl!Allocator. With 
such IAllocatorImpl, user code can specify concrete expectations 
about the allocator without needing to know the exact allocator 
type.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 04/30/2017 05:39 PM, Andrei Alexandrescu wrote:

The allocators design is interesting in that it has a full DbI core, on
top of which resides a thin dynamically-type interface (IAllocator and
ISharedAllocator). We're still exploring the idioms enabled by this
interaction. -- Andrei


I assume this dual-interface design (DbI + OO) is, much like std.digest, 
a way to have our template/DbI cake and still permit things to be 
selected at runtime too (via OO), right?


If so, this is a topic that's been on my mind lately, and I'd like to 
offer some thoughts I've had (well, some partial thoughts anyway):


Part A: While I like that the std.digest design allows both DbI and 
runtime-selection of arbitrary user-created types, it has a few issues:


1. It feels awkward that it needs essentially duplicate APIs under two 
different paradigms to pull it off.


2. If a library (such as mine) wants to support both versions, then it 
needs a little bit of extra plumbing to accommodate that. Or at least it 
seemed to in my case. (std.digest itself even has a little bit of extra 
plumbing because of it: WrapperDigest)


3. The OO version, being OO, does involve some type erasure.

Now, fast-forward to:

Part B: On a separate project, I've used Sonke's TaggedAlgebraic before 
<https://github.com/s-ludwig/taggedalgebraic>. It's kind of interesting: 
It's much like Phobos's Algebraic, except it allows function/member 
forwarding to the current underlying value *without* having to 
explicitly check what the current value's type is (Don't recall offhand 
exactly how it handles members that don't exist on the current type, 
probably throws I guess, although I'd imagine it would be possible to 
statically disallow access to members known at compile-time to not exist 
in *any* of the possible underlying types. But I digress).


But then:

Part C: At the time, I saw TaggedAlgebraic's member-forwarding as a 
minor little convenience feature. But the other day I was working with 
some code that used and built upon std.digest, and something occurred to me:


If we had a type similar to TaggedAlgebraic, but was an open variant 
rather than a closed algebraic (Ie, a type like Phobos's Variant, but 
forwarded member access without first requiring explicit conversion to 
the exact underlying type), then *that* could be used to permit 
runtime-selection between arbitrary DbI types *without* the type-erasure 
and "duality" of adding in OO (or at least, less type-erasure anyway). 
Just maybe need the ability to filter the allowable types via a 
constraint (such as isBidirctionalRange, etc), and then, IIUC it should 
be possible for the variant-like supertype to qualify as an 
honest-to-goodness instance of the DbI interface. Voila: OO has been 
obsoleted.


Destroy?

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Andrei Alexandrescu via Digitalmars-d]

On 05/01/2017 08:12 AM, Guillaume Piolat wrote:

On Sunday, 30 April 2017 at 21:43:26 UTC, Andrei Alexandrescu wrote:


A pass through the root allocators (Mallocator, GCAllocator etc)
figuring out what attributes could be meaningfully attached would be
welcome. The rest would rely on inference.


Thanks,

Andrei


IAllocator being fully @nogc would be a comforting guarantee, as runtime
dispatch makes for lighter types.


As I said (and am encouraging you to do this), this is achieved through 
simple variance:


interface INoGCAllocator : IAllocator {
   ... override all methods here as @nogc ...
}

This is possible because a @nogc method may override one that is not 
@nogc. @nogc being more restrictive it is contravariant with no-@nogc.


Also IAllocator should have a few @nogc methods to start with; there's 
no reason e.g. for empty() to create garbage.


Could you please initiate a PR?


Andrei

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Guillaume Piolat via Digitalmars-d]

On Sunday, 30 April 2017 at 21:43:26 UTC, Andrei Alexandrescu 
wrote:


A pass through the root allocators (Mallocator, GCAllocator 
etc) figuring out what attributes could be meaningfully 
attached would be welcome. The rest would rely on inference.



Thanks,

Andrei


IAllocator being fully @nogc would be a comforting guarantee, as 
runtime dispatch makes for lighter types.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Monday, 1 May 2017 at 04:54:28 UTC, Andrei Alexandrescu wrote:

On 4/30/17 8:43 PM, Stanislav Blinov wrote:
On Sunday, 30 April 2017 at 21:43:26 UTC, Andrei Alexandrescu 
wrote:

On 04/27/2017 07:35 PM, Stanislav Blinov wrote:
IAllocator is too high level an interface, it doesn't carry 
any
information as to what type of memory it can allocate (so we 
can only
assume unshared), and does or does it not use GC (so we can 
only assume

GC).


Initially all fresh memory is unshared. Whether or not the 
user subsequently shares it is of no consequence to the 
allocator.


Why would we need any ISharedAllocator then?


The allocator itself may be shared across threads...


But it is no different in case of memory. Allocator that 
allocates shared memory should either return shared(void)[] or 
have an explicit interface (allocateShared et al.), not just have 
the  "you may cast result of allocate() to shared" somewhere in 
the docs. We should let the language do the work it can, not send 
us to RTFM. Hence my earlier statement that more interfaces are 
needed, as one possible (albeit not pretty) solution.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Sebastiaan Koppe via Digitalmars-d]

On Friday, 28 April 2017 at 22:18:54 UTC, Atila Neves wrote:
Done. I also added to the README that it has its own versions 
of the range constraints from Phobos that can be used with 
`@models`.


Atila


Example of an error message in the README would be great too.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Andrei Alexandrescu via Digitalmars-d]

On 4/30/17 8:43 PM, Stanislav Blinov wrote:

On Sunday, 30 April 2017 at 21:43:26 UTC, Andrei Alexandrescu wrote:

On 04/27/2017 07:35 PM, Stanislav Blinov wrote:

IAllocator is too high level an interface, it doesn't carry any
information as to what type of memory it can allocate (so we can only
assume unshared), and does or does it not use GC (so we can only assume
GC).


Initially all fresh memory is unshared. Whether or not the user 
subsequently shares it is of no consequence to the allocator.


Why would we need any ISharedAllocator then?


The allocator itself may be shared across threads, in which case its 
primitives need to be reentrant. -- Andrei

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Monday, 1 May 2017 at 00:43:22 UTC, Stanislav Blinov wrote:


block_ = allocator_.allocate(T.sizeof);


Obviously, should be Block.sizeof, and


AllocatorInterface!AllocTraits allocator_;


should be AllocatorInterface!traits allocator_

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Sunday, 30 April 2017 at 21:43:26 UTC, Andrei Alexandrescu 
wrote:

On 04/27/2017 07:35 PM, Stanislav Blinov wrote:

IAllocator is too high level an interface, it doesn't carry any
information as to what type of memory it can allocate (so we 
can only
assume unshared), and does or does it not use GC (so we can 
only assume

GC).


Initially all fresh memory is unshared. Whether or not the user 
subsequently shares it is of no consequence to the allocator.


Why would we need any ISharedAllocator then? If we're to force 
users to rely on *documentation* whether or not they can cast 
allocated memory to shared, there is no point in static checks, 
they'll only get in the way of user's attempts to try and stitch 
together the pieces. They're on their own anyway at that point.


If we are to devise types with allocators as members instead 
of type
arguments, we need the additional information. Better to catch 
invalid
assignment at compile time than to chase down how a stack 
allocator from

one thread ended up in another.


A pass through the root allocators (Mallocator, GCAllocator 
etc) figuring out what attributes could be meaningfully 
attached would be welcome. The rest would rely on inference.


If we're type-erasing allocators, we have to have the ability to 
statically specify what traits we're interested in (i.e. *don't* 
want to erase). Otherwise, inference will not be of any help.
Consider an example, inspired by the discussion of Atila's 
automem library:


import std.experimental.allocator;
import std.algorithm.mutation : move;

// Allocator is not a type parameter
struct SmartPtr(T)
{
// we can infer all we need form A...
this(A, Args...)(A alloc, auto ref Args args)
if (isAllocatorImpl!A)
{
// ...but we immediately throw the inference away:
allocator_ = alloc;
block_ = cast(Block[])allocator_.allocate(Block.sizeof);
// SmartPtr logic snipped...
}

~this()
{
// ...SmartPtr logic snipped
allocator_.deallocate(block_);
}

private:
struct Block
{
size_t refCount;
void[T.sizeof] payload;
}

Block[] block_;
IAllocator allocator_;
}

struct Data {
~this() @nogc { /*...*/ }
// the rest of implementation is @nogc as well
}

struct MyType
{
// won't compile: IAllocator's methods aren't @nogc,
// so SmartPtr's dtor isn't either
this(SmartPtr!Data data) @nogc
{
data_ = move(data);
}

private:
SmartPtr!Data data_;
}

Obviously, IAllocator is not the tool for the job here. This 
calls for something like this:


enum AllocTraits
{
none  = 0x00,
nogc  = 0x01,
share = 0x02
}

alias AllocatorInterface(AllocTraits) = // ???

struct SmartPtr(T, AllocTraits traits = AllocTraits.none)
{
this(A, Args...)(A alloc, auto ref Args args)
{
// this should not compile if A isn't compatible
// with `traits`:
allocator_ = alloc;
block_ = allocator_.allocate(T.sizeof);
// SmartPtr logic snipped...
}

~this()
{
// ...SmartPtr logic snipped
allocator_.deallocate(block_);
}

private:
void[] block_;
AllocatorInterface!AllocTraits allocator_;
}

alias DataPtr = SmartPtr!(Data, AllocTraits.nogc);

struct MyType
{
this(DataPtr data) @nogc
{
data_ = move(data);
}

private:
DataPtr data_;
}

That *would* be able to rely on inference. Question is, what is 
AllocatorInterface? Should we push such type erasure to the users?

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Andrei Alexandrescu via Digitalmars-d]

On 04/27/2017 07:35 PM, Stanislav Blinov wrote:

IAllocator is too high level an interface, it doesn't carry any
information as to what type of memory it can allocate (so we can only
assume unshared), and does or does it not use GC (so we can only assume
GC).


Initially all fresh memory is unshared. Whether or not the user 
subsequently shares it is of no consequence to the allocator. Allocators 
must, however, distinguish between reentrant and non-reentrant calls to 
the allocation primitives.


With regard to whether the memory is collectable or not we need some 
more thinking.



If we are to devise types with allocators as members instead of type
arguments, we need the additional information. Better to catch invalid
assignment at compile time than to chase down how a stack allocator from
one thread ended up in another.


A pass through the root allocators (Mallocator, GCAllocator etc) 
figuring out what attributes could be meaningfully attached would be 
welcome. The rest would rely on inference.



Thanks,

Andrei

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Andrei Alexandrescu via Digitalmars-d]

On 04/27/2017 07:12 PM, Moritz Maxeiner wrote:

On Thursday, 27 April 2017 at 20:04:32 UTC, Stanislav Blinov wrote:

On Thursday, 27 April 2017 at 19:57:52 UTC, Andrei Alexandrescu wrote:

https://github.com/dlang/phobos/pull/5355

Andrei


And then we'd probably need INoGCAllocator and ISharedNOGCAllocator...


Wasn't one major selling point of compile time introspection / duck
typing that we could stop using interfaces such... naming schemes?


The allocators design is interesting in that it has a full DbI core, on 
top of which resides a thin dynamically-type interface (IAllocator and 
ISharedAllocator). We're still exploring the idioms enabled by this 
interaction. -- Andrei

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Andrei Alexandrescu via Digitalmars-d]

On 04/27/2017 04:04 PM, Stanislav Blinov wrote:

On Thursday, 27 April 2017 at 19:57:52 UTC, Andrei Alexandrescu wrote:

https://github.com/dlang/phobos/pull/5355

Andrei


And then we'd probably need INoGCAllocator and ISharedNOGCAllocator...


"shared" is a type qualifier essentially changing the type of the 
interface, so a separate interface is needed. For @nogc, simple variance 
is sufficient:


interface A
{
void fun();
}

class B : A
{
void fun() @nogc {}
}

That said, there are quite a few functions we should be able to qualify 
as @nogc from the get-go, e.g. empty, expand, alignment etc. Would you 
like to try a PR to that effect? Thanks!



Andrei

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Atila Neves via Digitalmars-d]

On Friday, 28 April 2017 at 18:04:06 UTC, jmh530 wrote:

On Friday, 28 April 2017 at 17:09:22 UTC, Atila Neves wrote:


https://github.com/atilaneves/concepts

import concepts;

@models!(isForwardRange, MyType)
struct MyType {  }


Atila


I remember you had posted about this last year. It looks like 
you've added some stuff on ranges to it recently...interesting.


You might want to include an example in the Readme.md of using 
the concept as a template constraint for a function. Maybe like


void callfoo(T)(T x)
if (isFoo!T)
{
x.foo();
}

Foo foo;
callfoo(foo);


Done. I also added to the README that it has its own versions of 
the range constraints from Phobos that can be used with `@models`.


Atila

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[jmh530 via Digitalmars-d]

On Friday, 28 April 2017 at 17:09:22 UTC, Atila Neves wrote:


https://github.com/atilaneves/concepts

import concepts;

@models!(isForwardRange, MyType)
struct MyType {  }


Atila


I remember you had posted about this last year. It looks like 
you've added some stuff on ranges to it recently...interesting.


You might want to include an example in the Readme.md of using 
the concept as a template constraint for a function. Maybe like


void callfoo(T)(T x)
if (isFoo!T)
{
x.foo();
}

Foo foo;
callfoo(foo);

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 04/28/2017 01:09 PM, Atila Neves wrote:


https://github.com/atilaneves/concepts

import concepts;

@models!(isForwardRange, MyType)
struct MyType {  }



Hmm, close, but if I'm reading the source right, it looks like a type 
doesn't have to use the UDA in order for isXXX or "static 
assert(models..." to be satisfied, which was the whole issue I had. 
(Also, it'd be nice if it didn't require both isXXX and checkXXX to be 
manually written, because given a checkXXX function, the isXXX function 
is just boilerplate.)


Although I guess the checkXXX function could simply include a check the 
the UDA. Be nice if that was taken care of automatically though. Now if 
only we could get Phobos to do that...


I do, however, like the improved diagnostics this offers, and the fact 
that the checkXXX function uses a much saner (simpler, easier to 
read/remember/understand) syntax than the usual `is(typeof((){ ... })))` 
mess. I may use this.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Atila Neves via Digitalmars-d]

On Friday, 28 April 2017 at 01:12:39 UTC, Nick Sabalausky 
(Abscissa) wrote:

On 04/27/2017 07:12 PM, Moritz Maxeiner wrote:


Wasn't one major selling point of compile time introspection / 
duck
typing that we could stop using interfaces such... naming 
schemes?


Not that I speak for everyone, but the way I see it, no, the 
duck typing aspect is just something that's to be tolerated. 
The real gold is simply being freed from both the rigid mess 
and the runtime costs of class/interface hierarchies. I wish we 
DID have to include some kind of "implements ForwardRange" or 
"enum _thisStructImplements_ForwardRange" or some such to 
declare a type actually intends to be a ForwardRange (or 
whatever) and isn't merely qualifying as one by coincidence.


https://github.com/atilaneves/concepts

import concepts;

@models!(isForwardRange, MyType)
struct MyType {  }


Atila

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 04/27/2017 07:12 PM, Moritz Maxeiner wrote:


Wasn't one major selling point of compile time introspection / duck
typing that we could stop using interfaces such... naming schemes?


Not that I speak for everyone, but the way I see it, no, the duck typing 
aspect is just something that's to be tolerated. The real gold is simply 
being freed from both the rigid mess and the runtime costs of 
class/interface hierarchies. I wish we DID have to include some kind of 
"implements ForwardRange" or "enum _thisStructImplements_ForwardRange" 
or some such to declare a type actually intends to be a ForwardRange (or 
whatever) and isn't merely qualifying as one by coincidence.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Thursday, 27 April 2017 at 23:12:48 UTC, Moritz Maxeiner wrote:
On Thursday, 27 April 2017 at 20:04:32 UTC, Stanislav Blinov 
wrote:
On Thursday, 27 April 2017 at 19:57:52 UTC, Andrei 
Alexandrescu wrote:

https://github.com/dlang/phobos/pull/5355

Andrei


And then we'd probably need INoGCAllocator and 
ISharedNOGCAllocator...


Wasn't one major selling point of compile time introspection / 
duck typing that we could stop using interfaces such... naming 
schemes?


IAllocator is too high level an interface, it doesn't carry any 
information as to what type of memory it can allocate (so we can 
only assume unshared), and does or does it not use GC (so we can 
only assume GC).


If we are to devise types with allocators as members instead of 
type arguments, we need the additional information. Better to 
catch invalid assignment at compile time than to chase down how a 
stack allocator from one thread ended up in another.

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Moritz Maxeiner via Digitalmars-d]

On Thursday, 27 April 2017 at 20:04:32 UTC, Stanislav Blinov 
wrote:
On Thursday, 27 April 2017 at 19:57:52 UTC, Andrei Alexandrescu 
wrote:

https://github.com/dlang/phobos/pull/5355

Andrei


And then we'd probably need INoGCAllocator and 
ISharedNOGCAllocator...


Wasn't one major selling point of compile time introspection / 
duck typing that we could stop using interfaces such... naming 
schemes?

Re: Interesting PRs: bringing type system legitimacy to shared allocators

[Stanislav Blinov via Digitalmars-d]

On Thursday, 27 April 2017 at 19:57:52 UTC, Andrei Alexandrescu 
wrote:

https://github.com/dlang/phobos/pull/5355

Andrei


And then we'd probably need INoGCAllocator and 
ISharedNOGCAllocator...

Interesting PRs: bringing type system legitimacy to shared allocators

[Andrei Alexandrescu via Digitalmars-d]

https://github.com/dlang/phobos/pull/5355

Andrei

Re: Allocators and Containers

[Jon Degenhardt via Digitalmars-d]

On Friday, 17 February 2017 at 04:13:06 UTC, Jack Stouffer wrote:
This is based on conversations I've had with Andrei and Walter 
on here and on Github.


There was a bunch of threads here about six months ago by 
Andrei talking about the different problems he was running into 
when trying to implement @safe and @nogc containers. One of the 
conclusions was that ref counting was going to have to be a 
nessesary component of the solution. This led to DIP1000 
because of Andrei's conclusion that "Safe reference counting 
cannot be implemented as a library" 
https://forum.dlang.org/post/n0nnu0$1tth$1...@digitalmars.com.


Also, there's the vision document, which lays out some goals 
but does not talk about containers: 
https://wiki.dlang.org/Vision/2017H1


Excellent, thanks for sharing this.  --Jon

Re: Allocators and Containers

[Jack Stouffer via Digitalmars-d]

On Friday, 17 February 2017 at 03:05:04 UTC, Jon Degenhardt wrote:
This is very useful information. Is there a thread or document 
describing this further? I'm not looking for hard decisions. 
Just that it's useful to have bigger picture view of what is on 
the horizon. If there's nothing readily available, perhaps a 
talk at DConf?


--Jon


This is based on conversations I've had with Andrei and Walter on 
here and on Github.


There was a bunch of threads here about six months ago by Andrei 
talking about the different problems he was running into when 
trying to implement @safe and @nogc containers. One of the 
conclusions was that ref counting was going to have to be a 
nessesary component of the solution. This led to DIP1000 because 
of Andrei's conclusion that "Safe reference counting cannot be 
implemented as a library" 
https://forum.dlang.org/post/n0nnu0$1tth$1...@digitalmars.com.


Also, there's the vision document, which lays out some goals but 
does not talk about containers: 
https://wiki.dlang.org/Vision/2017H1

Re: Allocators and Containers

[Jon Degenhardt via Digitalmars-d]

On Thursday, 16 February 2017 at 15:48:44 UTC, Jack Stouffer 
wrote:
The current std.containers design was not designed with 
allocators in mind. The current plan is


1. DIP1000, which adds safety checks for escape analysis to the 
language, must be completely implemented in order to have @safe 
containers
2. A new containers design will be submitted to 
std.experimental to eventually replace the current one


This is very useful information. Is there a thread or document 
describing this further? I'm not looking for hard decisions. Just 
that it's useful to have bigger picture view of what is on the 
horizon. If there's nothing readily available, perhaps a talk at 
DConf?


--Jon

Re: Allocators and Containers

[Basile B. via Digitalmars-d]

On Thursday, 16 February 2017 at 15:37:52 UTC, Minty Fresh wrote:
A lot of the usefulness of the std.experimental.allocators 
module is lost because no other part of the stdlib actually 
ties into the functionality provided by it.


For example, the Array type defined in std.container relies on 
malloc() directly, so if you wanted to use a type to replace 
built-in arrays with a custom allocator, you'd need to 
implement your own container type.


Would it make sense to allow the std.container types to accept 
IAllocator instances, and to allow custom allocators? (Using 
Mallocator by default.)


Appender also, although there some WIP for a complete @nogc 
version (which I'd prefer to be based on allocators too, but well 
not possible b/c of the reasons mentioned above).

Re: Allocators and Containers

[Seb via Digitalmars-d]

On Thursday, 16 February 2017 at 15:48:44 UTC, Jack Stouffer 
wrote:
On Thursday, 16 February 2017 at 15:37:52 UTC, Minty Fresh 
wrote:
A lot of the usefulness of the std.experimental.allocators 
module is lost because no other part of the stdlib actually 
ties into the functionality provided by it.


The current rule of the standard library is that stuff outside 
of experimental is not allowed to use experimental, because, 
well, it's experimental. It's likely to break.


Yep, for reference please see this thread: 
http://forum.dlang.org/post/hxwadyrnvzvutrdcg...@forum.dlang.org


tld;dr: Current blockers are:
- no CTFE support
- dependent on druntime

Re: Allocators and Containers

[Jack Stouffer via Digitalmars-d]

On Thursday, 16 February 2017 at 15:37:52 UTC, Minty Fresh wrote:
A lot of the usefulness of the std.experimental.allocators 
module is lost because no other part of the stdlib actually 
ties into the functionality provided by it.


The current rule of the standard library is that stuff outside of 
experimental is not allowed to use experimental, because, well, 
it's experimental. It's likely to break.


For example, the Array type defined in std.container relies on 
malloc() directly, so if you wanted to use a type to replace 
built-in arrays with a custom allocator, you'd need to 
implement your own container type.


Would it make sense to allow the std.container types to accept 
IAllocator instances, and to allow custom allocators? (Using 
Mallocator by default.)


The current std.containers design was not designed with 
allocators in mind. The current plan is


1. DIP1000, which adds safety checks for escape analysis to the 
language, must be completely implemented in order to have @safe 
containers
2. A new containers design will be submitted to std.experimental 
to eventually replace the current one


In the mean time you can always use 
https://code.dlang.org/packages/emsi_containers

Allocators and Containers

[Minty Fresh via Digitalmars-d]

A lot of the usefulness of the std.experimental.allocators module 
is lost because no other part of the stdlib actually ties into 
the functionality provided by it.


For example, the Array type defined in std.container relies on 
malloc() directly, so if you wanted to use a type to replace 
built-in arrays with a custom allocator, you'd need to implement 
your own container type.


Would it make sense to allow the std.container types to accept 
IAllocator instances, and to allow custom allocators? (Using 
Mallocator by default.)

Re: Long term @nogc / allocators usage

[Dicebot via Digitalmars-d]

Also in general usage of custom allocator for temporaries is a 
very bad @nogc approach and is only marginally better than GC 
using code. Good @nogc must make no decisions about allocations 
at all (and ideally minimize stack allocations too, especially if 
fibers are to be used).

Re: Long term @nogc / allocators usage

[Dicebot via Digitalmars-d]

@nogc code is quite different from idiomatic "casual" D code and 
one is expected to make sacrifices to go for it. Here are some 
approaches I decided for myself:


On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta wrote:
Now, there are some cases in which you cannot avoid the managed 
allocations:
1) throwing exceptions: these should not be abandoned in favor 
of other solutions; IMHO, they should be easily usable in @nogc 
code; switching to error codes or user-defined callbacks is not 
feasible in general;


Use pre-allocated exception instances. Throwing itself doesn't 
require GC, only allocating exception does. You can possibly 
screw up exception chaining this way but this is a very minor 
loss.


2) returning arrays: sometimes you just can't avoid this: if 
your function must return a string, than it has to allocate it 
(unless it's a slice of some input)


If it can't be avoided, this is not @nogc code and there is no 
point in pretending otherwise. Use ranges and sinks instead to 
move allocations decisions up the call chain. Note that with sink 
approach you want be able to mark function itself @nogc (because 
sink delegate allocates) but you can mark unittest block that 
verifies there are no _other_ allocations:


void foo ( alias sink_dg ) ( )
{
sink_dg("abc");
}

@nogc unittest
{
void noop_sink ( const(char)[] chunk ) @nogc { }
foo!noop_sink();
}

Re: Long term @nogc / allocators usage

[ZombineDev via Digitalmars-d]

On Sunday, 24 July 2016 at 17:04:43 UTC, Seb wrote:
On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta 
wrote:
DISCLAIMER: although I've been occasionally using D for a lot 
of time, I only recently started to follow the forums and use 
it intensively, so I may miss part of the history / long term 
vision.


So, the current guidelines to make a function usable in @nogc 
are:

1) use ranges as much as possible, instead of arrays;
2) don't specify a precise delegate type, but make it 
templated (so you accept both gc and @nogc delegates);
3) use manual management (e.g. malloc/free) for internal 
buffers whose lifetime and ownership are easy to track.


Now, there are some cases in which you cannot avoid the 
managed allocations:
1) throwing exceptions: these should not be abandoned in favor 
of other solutions; IMHO, they should be easily usable in 
@nogc code; switching to error codes or user-defined callbacks 
is not feasible in general;
2) returning arrays: sometimes you just can't avoid this: if 
your function must return a string, than it has to allocate it 
(unless it's a slice of some input)


With the new allocators library, we can customize these 
allocations. There are two possible ways to follow, both with 
advantages and drawbacks:
1) have all allocating functions take a templated allocator 
parameter (defaulting to GCAllocator) and use it to allocate 
returned arrays and thrown exceptions; this allows the 
compiler to infer @nogc whenever a @nogc allocator is passed, 
but becomes bloated because you have to carry around another 
parameter to lots of functions


When I had to deal with this problem, I liked this idea too, but

1) Allocations with GCAllocator (e.g. makeArray) are neither 
@safe, nothrow nor pure.

2) It creates a huge template bloat

For 1) the best solution is WIP here:

https://github.com/dlang/phobos/pull/3891

2) have all allocating functions use theAllocator instead of 
raw new to perform allocations. This would make the allocator 
parameter implicit and the code very easy (just set 
theAllocator on startup), but would not allow the compiler to 
infer @nogc; IMHO it's not that bad: you can always use the 
profiler to check that your code is in fact @nogc, even if not 
stated explicitly; but many will not agree with this.


Yep it destroys the point of @nogc, besides different data 
structures / algorithms profit from different, specialized 
allocators.


So my question is: what's the plan? Which road is to be 
followed? How will Phobos evolve regarding this?


There are also other options [1], e.g.
- using `static if`
- using two separate functions (one with `new`)
- requiring the API user to allocate the data beforehand

[1] https://github.com/dlang/phobos/pull/4190


I prefer alias template parameters as they provide maximum 
flexibility:

https://github.com/dlang/phobos/pull/4288#issuecomment-227609141

Re: Long term @nogc / allocators usage

[Seb via Digitalmars-d]

On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta wrote:
DISCLAIMER: although I've been occasionally using D for a lot 
of time, I only recently started to follow the forums and use 
it intensively, so I may miss part of the history / long term 
vision.


So, the current guidelines to make a function usable in @nogc 
are:

1) use ranges as much as possible, instead of arrays;
2) don't specify a precise delegate type, but make it templated 
(so you accept both gc and @nogc delegates);
3) use manual management (e.g. malloc/free) for internal 
buffers whose lifetime and ownership are easy to track.


Now, there are some cases in which you cannot avoid the managed 
allocations:
1) throwing exceptions: these should not be abandoned in favor 
of other solutions; IMHO, they should be easily usable in @nogc 
code; switching to error codes or user-defined callbacks is not 
feasible in general;
2) returning arrays: sometimes you just can't avoid this: if 
your function must return a string, than it has to allocate it 
(unless it's a slice of some input)


With the new allocators library, we can customize these 
allocations. There are two possible ways to follow, both with 
advantages and drawbacks:
1) have all allocating functions take a templated allocator 
parameter (defaulting to GCAllocator) and use it to allocate 
returned arrays and thrown exceptions; this allows the compiler 
to infer @nogc whenever a @nogc allocator is passed, but 
becomes bloated because you have to carry around another 
parameter to lots of functions


When I had to deal with this problem, I liked this idea too, but

1) Allocations with GCAllocator (e.g. makeArray) are neither 
@safe, nothrow nor pure.

2) It creates a huge template bloat

For 1) the best solution is WIP here:

https://github.com/dlang/phobos/pull/3891

2) have all allocating functions use theAllocator instead of 
raw new to perform allocations. This would make the allocator 
parameter implicit and the code very easy (just set 
theAllocator on startup), but would not allow the compiler to 
infer @nogc; IMHO it's not that bad: you can always use the 
profiler to check that your code is in fact @nogc, even if not 
stated explicitly; but many will not agree with this.


Yep it destroys the point of @nogc, besides different data 
structures / algorithms profit from different, specialized 
allocators.


So my question is: what's the plan? Which road is to be 
followed? How will Phobos evolve regarding this?


There are also other options [1], e.g.
- using `static if`
- using two separate functions (one with `new`)
- requiring the API user to allocate the data beforehand

[1] https://github.com/dlang/phobos/pull/4190

Re: Long term @nogc / allocators usage

[Lodovico Giaretta via Digitalmars-d]

On Sunday, 24 July 2016 at 15:44:48 UTC, lqjglkqjsg wrote:
Note that for the arrays you have @nogc routines in 
std.experimental.allocators. (expand/shrink can infer @nogc 
from Mallocator).


I know. In fact I'm proposing two possible solutions based on 
std.experimental.allocators. Both have advantages and drawbacks, 
so I'm asking which one shall be followed, which one will be used 
my Phobos, or if a third solution exists.

Re: Long term @nogc / allocators usage

[lqjglkqjsg via Digitalmars-d]

On Sunday, 24 July 2016 at 15:34:55 UTC, Lodovico Giaretta wrote:
DISCLAIMER: although I've been occasionally using D for a lot 
of time, I only recently started to follow the forums and use 
it intensively, so I may miss part of the history / long term 
vision.


[...]


Note that for the arrays you have @nogc routines in 
std.experimental.allocators. (expand/shrink can infer @nogc from 
Mallocator).

Long term @nogc / allocators usage

[Lodovico Giaretta via Digitalmars-d]

DISCLAIMER: although I've been occasionally using D for a lot of 
time, I only recently started to follow the forums and use it 
intensively, so I may miss part of the history / long term vision.


So, the current guidelines to make a function usable in @nogc are:
1) use ranges as much as possible, instead of arrays;
2) don't specify a precise delegate type, but make it templated 
(so you accept both gc and @nogc delegates);
3) use manual management (e.g. malloc/free) for internal buffers 
whose lifetime and ownership are easy to track.


Now, there are some cases in which you cannot avoid the managed 
allocations:
1) throwing exceptions: these should not be abandoned in favor of 
other solutions; IMHO, they should be easily usable in @nogc 
code; switching to error codes or user-defined callbacks is not 
feasible in general;
2) returning arrays: sometimes you just can't avoid this: if your 
function must return a string, than it has to allocate it (unless 
it's a slice of some input)


With the new allocators library, we can customize these 
allocations. There are two possible ways to follow, both with 
advantages and drawbacks:
1) have all allocating functions take a templated allocator 
parameter (defaulting to GCAllocator) and use it to allocate 
returned arrays and thrown exceptions; this allows the compiler 
to infer @nogc whenever a @nogc allocator is passed, but becomes 
bloated because you have to carry around another parameter to 
lots of functions
2) have all allocating functions use theAllocator instead of raw 
new to perform allocations. This would make the allocator 
parameter implicit and the code very easy (just set theAllocator 
on startup), but would not allow the compiler to infer @nogc; 
IMHO it's not that bad: you can always use the profiler to check 
that your code is in fact @nogc, even if not stated explicitly; 
but many will not agree with this.


So my question is: what's the plan? Which road is to be followed? 
How will Phobos evolve regarding this?

[Issue 15775] Object.factory replacement based on experimental allocators

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15775

b2.t...@gmx.com changed:

   What|Removed |Added

 Status|ASSIGNED|RESOLVED
 Resolution|--- |REMIND

--

Re: Mir v0.13.0: mir.combinatorics & mir.ndslice allocators/constructors/converters

[Joseph Rushton Wakeling via Digitalmars-d-announce]

On Monday, 11 April 2016 at 18:20:29 UTC, 9il wrote:

Mir v0.13.0 was released.

http://mir.dlang.io


Interesting choice of project name there -- for a second I 
thought you were providing display server bindings ;-)

Mir v0.13.0: mir.combinatorics & mir.ndslice allocators/constructors/converters

[9il via Digitalmars-d-announce]

Mir v0.13.0 was released.

http://mir.dlang.io

# New functions was added:

- `mir.combinatorics` (by Sebastian Wilzbach)
  - `combinations`
  - `combinationsRepeat`
  - `permutations`
  - `cartesianPower`

- `mir.ndslice.slice`
 - `SliceException`
 - `shape`
 - `slice`
 - `makeSlice`
 - `ndarray`
 - `makeNdarray`

- `mir.ndslice.selection`
 - `iotaSlice`/`IotaSlice`

For corresponding ndslice updates in Phobos see
https://github.com/D-Programming-Language/phobos/pull/4182

Best regards,
Ilya

Typed D allocators based on Google Thread caching memory allocators

[Basile B. via Digitalmars-d-announce]

This new DUB package uses the "Thread Caching" memory allocators 
from Google performance tools to make typed D allocators, conform 
with the interface defined in std.experimental.allocator.


- Posix only.
- TCMallocator (= Mallocator).
- TCAlignedMallocator (= AlignedMallocator).

And maybe more (TCMMapAllocator TCCAllocator).

about Thread caching memory allocators:
http://goog-perftools.sourceforge.net/doc/tcmalloc.html

the repo:
https://github.com/BBasile/tcmallocd

PR and BR obviously welcome if you see an error.

Re: Containers, Allocators and Purity

[ZombineDev via Digitalmars-d-learn]

On Thursday, 24 March 2016 at 11:18:06 UTC, Nordlöw wrote:
Could somebody briefly outline how the thread-locality 
(non-GC-locked) of allocators  relates to the purity of the 
containers using them?


This because I want to move forward with optimizations in my 
knowledge graph that requires GC-free array containers storing 
value typed elements (integers) which preferrably has pure API.


Specifically, I want to use something like

https://github.com/economicmodeling/containers/blob/master/src/containers/dynamicarray.d

that is `@safe pure` and uses lock-free allocations in a 
heavily multi-threaded application.


If I want purity aswell which `std.experimental.allocators` are 
possible?


After a quick glance at: 
https://github.com/economicmodeling/containers/blob/master/src/containers/dynamicarray.d


It looks like if `useGC == false`, all of the methods should be 
`pure`-ready, except for `remove(size_t)` and `~this()`. These 
two methods use `typeid.destroy`, which I don't think is `pure`.

Re: Containers, Allocators and Purity

[ZombineDev via Digitalmars-d-learn]

On Thursday, 24 March 2016 at 11:18:06 UTC, Nordlöw wrote:
Could somebody briefly outline how the thread-locality 
(non-GC-locked) of allocators  relates to the purity of the 
containers using them?


This because I want to move forward with optimizations in my 
knowledge graph that requires GC-free array containers storing 
value typed elements (integers) which preferrably has pure API.


Specifically, I want to use something like

https://github.com/economicmodeling/containers/blob/master/src/containers/dynamicarray.d

that is `@safe pure` and uses lock-free allocations in a 
heavily multi-threaded application.


If I want purity aswell which `std.experimental.allocators` are 
possible?


Currently almost nothing in `std.experimental.allocators` is 
explicitly marked as pure, except for some of the things in 
./common.d 
(https://github.com/D-Programming-Language/phobos/pull/3957 
*shameless plug*). Thankfully make, makeArray, dispose and some 
of the allocators are templates, so you can rely on attribute 
inference.


The most important thing is that you need to have pure `allocate` 
and `deallocate` methods. After this `make` and friends will work 
like magic (at least for types with pure `this` and `~this`). 
`pure` statefull allocator essentially means that it has to be 
thread-local, so there are no global side-effects. Essentially, 
to achieve this you can make a large initial allocation in each 
thread (even from a non-pure allocator) and then you can make 
`pure` sub-allocations out of it.


You can read more about my adventures with my `pure` smart 
pointer here:

http://forum.dlang.org/post/eegjluaiwvdxfnbxk...@forum.dlang.org
http://forum.dlang.org/post/bvgyrfvuqrqcyvhkq...@forum.dlang.org

Containers, Allocators and Purity

[Nordlöw via Digitalmars-d-learn]

Could somebody briefly outline how the thread-locality 
(non-GC-locked) of allocators  relates to the purity of the 
containers using them?


This because I want to move forward with optimizations in my 
knowledge graph that requires GC-free array containers storing 
value typed elements (integers) which preferrably has pure API.


Specifically, I want to use something like

https://github.com/economicmodeling/containers/blob/master/src/containers/dynamicarray.d

that is `@safe pure` and uses lock-free allocations in a heavily 
multi-threaded application.


If I want purity aswell which `std.experimental.allocators` are 
possible?

[Issue 15775] New: Object.factory replacement based on experimental allocators

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15775

  Issue ID: 15775
   Summary: Object.factory replacement based on experimental
allocators
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: b2.t...@gmx.com

Object.factory always allocates in the GC-heap. 

We could imagine a replacment that uses std.experimental.allocators to allow
something like a factory that would be manually managed or that would use a
free list, according to the user needs.

--

[Issue 15197] Allow allocator_list to work with allocators that do not support `expand()`

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15197

--- Comment #4 from github-bugzi...@puremagic.com ---
Commit pushed to stable at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/2408a31a3f08208194b7345aaeee64e7369010fc
Merge pull request #3689 from economicmodeling/allocator-list-without-expand

--

[Issue 15187] dispose for allocators is broken

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15187

--- Comment #5 from github-bugzi...@puremagic.com ---
Commits pushed to stable at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/bf64ff48a1821fbe9313fc930ed70f2d9b1bab8a
[DMD 2.069][allocators] Fix issue 15187

https://github.com/D-Programming-Language/phobos/commit/f2be4e5e1961ca259497ba0128ec70139b28d088
Merge pull request #3711 from 9il/patch-3

--

[Issue 3517] Allocators proposal

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=3517

Andrei Alexandrescu  changed:

   What|Removed |Added

 Status|ASSIGNED|RESOLVED
 Resolution|--- |FIXED

--- Comment #2 from Andrei Alexandrescu  ---
Finally can mark this as fixed.

--

[Issue 15187] dispose for allocators is broken

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15187

--- Comment #4 from github-bugzi...@puremagic.com ---
Commit pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/0422e29c7edd6fb807dcf878e78d1d3007a6e34b
Merge pull request #3711 from 9il/patch-3

--

[Issue 15197] Allow allocator_list to work with allocators that do not support `expand()`

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15197

--- Comment #3 from github-bugzi...@puremagic.com ---
Commit pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/275cad39c14a8b18ff7f141647c3b0820c8b14ac
Merge pull request #3689 from economicmodeling/allocator-list-without-expand

--

[Issue 15197] Allow allocator_list to work with allocators that do not support `expand()`

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15197

--- Comment #2 from github-bugzi...@puremagic.com ---
Commit pushed to stable at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/275cad39c14a8b18ff7f141647c3b0820c8b14ac
Merge pull request #3689 from economicmodeling/allocator-list-without-expand

Issue 15197 - Allow allocator_list to work with allocators that don't have
'expand()'

--

[Issue 15187] dispose for allocators is broken

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15187

--- Comment #3 from github-bugzi...@puremagic.com ---
Commit pushed to stable at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/0422e29c7edd6fb807dcf878e78d1d3007a6e34b
Merge pull request #3711 from 9il/patch-3

[DMD 2.069][allocators] Fix issue 15187

--

[Issue 15187] dispose for allocators is broken

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15187

--- Comment #2 from github-bugzi...@puremagic.com ---
Commits pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/bf64ff48a1821fbe9313fc930ed70f2d9b1bab8a
[DMD 2.069][allocators] Fix issue 15187

https://issues.dlang.org/show_bug.cgi?id=15187

Update package.d

https://github.com/D-Programming-Language/phobos/commit/f2be4e5e1961ca259497ba0128ec70139b28d088
Merge pull request #3711 from 9il/patch-3

[DMD 2.069][allocators] Fix issue 15187

--

[Issue 15197] Allow allocator_list to work with allocators that do not support `expand()`

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15197

--- Comment #1 from github-bugzi...@puremagic.com ---
Commit pushed to master at https://github.com/D-Programming-Language/phobos

https://github.com/D-Programming-Language/phobos/commit/2408a31a3f08208194b7345aaeee64e7369010fc
Merge pull request #3689 from economicmodeling/allocator-list-without-expand

Issue 15197 - Allow allocator_list to work with allocators that don't have
'expand()'

--

[Issue 15197] New: Allow allocator_list to work with allocators that do not support `expand()`

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15197

  Issue ID: 15197
   Summary: Allow allocator_list to work with allocators that do
not support `expand()`
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Keywords: pull
  Severity: enhancement
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: briancsch...@gmail.com

https://github.com/D-Programming-Language/phobos/pull/3689

--

[Issue 15187] dispose for allocators is broken

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15187

--- Comment #1 from Илья Ярошенко  ---
https://github.com/D-Programming-Language/phobos/pull/3711

--

[Issue 15187] New: dispose for allocators is broken

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15187

  Issue ID: 15187
   Summary: dispose for allocators is broken
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: major
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: ilyayaroshe...@gmail.com

Current code:

void dispose(A, T)(auto ref A alloc, T* p)
{
static if (hasElaborateDestructor!T)
{
destroy(*p);
}
alloc.deallocate(p[0 .. T.sizeof]);
}

Should be:

void dispose(A, T)(auto ref A alloc, T* p)
{
static if (hasElaborateDestructor!T)
{
destroy(*p);
}
alloc.deallocate((cast(void*)p)[0 .. T.sizeof]);
}

--

Re: Right after allocators: containers or database connectivity?

[Nick Sabalausky via Digitalmars-d]

On 06/09/2015 01:53 PM, Jacob Carlborg wrote:


I vote for databases. I also vote for an interface that is independent
of ODBC, but ODBC could be one of the drivers that implements this
interface. I know that there's a native implementation of the MySQL
protocol on code.dlang.org.



This is kind of my thought, too.

I haven't felt constrained by the lack of containers beyond what's 
built-in. And if I do, I can look into dcollections. It's arguably an 
optimization issue anyway. YES, Granted, algorithmic optimization is THE 
BIGGEST optimization by far, but aside from maybe big data domains it 
doesn't actually enable you to do things you just plain couldn't do at 
all otherwise.


Databases, OTOH, are something where you either can do it, or you just 
plain CAN'T. And with D it's currently a bit more on the can't side, 
especially if you don't want your app tied to just one DBMS. And unlike 
containers, the state of D databases actually HAS been an issue for me.


That said, tossing an ODBC wrapper in phobos doesn't really solve the DB 
issue. It would certainly be GOOD to have a good ODBC lib for D, no 
doubt, but:


1. There is apparently question as to whether it belongs in Phobos.

2. There's also question about async I/O. A *LOT* of DB users are also 
going to be Vibe.d users, so we cannot afford to have problems in this 
area. Perhaps what we need to look at is getting Vibe's async I/O (or at 
least some low-level portion of it, if that even makes sense?) into 
Phobos somehow?


3. As useful as ODBC may be, it still doesn't eliminate the need for 
direct drivers.


4. What's REALLY needed regarding DBs is a good, well-designed low-level 
interface capable of handling both ODBC and individual drivers. ORMs 
should be something separate, built on top of this later. This low-level 
part might either include NoSQL (if reasonable to do so) or leave NoSQL 
up to a separate interface (if need be).

Re: Right after allocators: containers or database connectivity?

[Andrei Alexandrescu via Digitalmars-d]

On 6/11/15 10:59 AM, Nick Sabalausky wrote:

2. There's also question about async I/O. A *LOT* of DB users are also
going to be Vibe.d users, so we cannot afford to have problems in this
area. Perhaps what we need to look at is getting Vibe's async I/O (or at
least some low-level portion of it, if that even makes sense?) into
Phobos somehow?


BTW I'm unclear about the ODBC async support on OSX and Linux, and 
Internet searches are not making that crystal clear. Does anyone know? 
-- Andrei

Re: Right after allocators: containers or database connectivity?

[Erik Smith via Digitalmars-d]

I’m actively working on the low level interface and I’m making 
good progress.  My goal is to have a high quality interface 
definition with working reference drivers for top tier databases. 
 The current references drivers are ODBC, sqlite, mysql, and 
Oracle.


I think it’s important to support ODBC since it covers all bases, 
but I agree that ODBC alone is not sufficient.  The main reason 
is that most application developers coming from other stacks will 
be expecting direct drivers and are not likely to be happy 
dealing with the added complications of ODBC.   However, I think 
both are achievable in the near term under a common interface.


On the async question, the native interfaces are traditionally 
synchronous and that’s my initial focus.   I will need to look at 
which native interfaces actually support it and see how that can 
be mapped into the generic interface, but synchronous needs to be 
supported.


There is a known issue with RefCounted that needs to be 
addressed.  There is a apparently a race condition with the 
reference counter when RefCounted is within a class.  Nearly all 
of the types in my design are ref counted structs and there are 
use cases for embedding those objects in classes.  There was 
discussion at the conference on this and I believe the preferred 
solution is for the GC have some thread affinity on collection.