Re: pure D mpeg2 decoder
On Friday, 24 June 2016 at 19:22:54 UTC, ketmar wrote: "pure D ffmpeg" dream is one step closer now. ;-) I already wrote it a couple of years ago with current but currently only transmuxing is supported with small set of containers: mpeg-ts and isom (mp4) =) I only need to publish it, but have to sort out some licensing problems.
pure D mpeg2 decoder
Hi all! I saw pure jpeg decoder was announced recently and I decided to publish pure D mpeg2 decoder that I wrote just for myself, with study aims. I didn't test it exhaustively, so don't judge me for bugs) Currently it supports only progressive sequences with no B frames. As for performance, it's 5 times slower than ffmpeg implementation, optimizations are required. link: https://github.com/theambient/mpeg2 Does someone want to write pure D AVC or HEVC decoder/encoder? =) P.S. It sometimes has some artifacts, I didn't try to fix them.
Re: Garbage collector collects live objects
On Thursday, 11 December 2014 at 18:36:59 UTC, Steven Schveighoffer wrote: My analysis so far: 2. In the array append code, the block attributes are obtained via GC.query, which has this code for getting the attributes: https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc.d#L1792 Quoting from that function: // reset the offset to the base pointer, otherwise the bits // are the bits for the pointer, which may be garbage offset = cast(size_t)(info.base - pool.baseAddr); info.attr = getBits(pool, cast(size_t)(offset pool.shiftBy)); Which should get the correct bits. I suspected there was an issue with getting the wrong bits, but this code looks correct. 3. The runtime caches the block info for thread local data for append speed. A potential issue is that the attributes are cached from a previous use for that block, but the GC (and the runtime itself) SHOULD clear that cache entry when that block is freed, avoiding this issue. A potential way to check this is to assert in a debug build of druntime that the cached block info always equals the actual block info. Are you able to build a debug version of druntime to test this? I can give you the changes you should make. This would explain the great difficulty in reproducing the issue. I will try to build debug version of dmd compiler and check the issue. 4. If your code is multi-threaded, but using __gshared, it can make the cache incorrect. Are you doing this? the app is multi-threaded via std.concurrency. there is only one known to me place where __gshared is used: logging library (checked by searching through whole source tree). make stub for this lib and try, so identify whether cache invalidated by _gshared or not. But the cache is really the only possible place I can see where the bits are set incorrectly, given that you just verified the bits are correct before the append. Can you just list the version of the compiler you are using? I want to make sure this isn't an issue that has already been fixed. the last. first of all i updated whole toolchain (dmd, dub). $ dmd DMD64 D Compiler v2.066.1 -Steve I started looking druntime and dmd source code myself before i checked the thread (thsnks for your help and feedback) and i have some questions. could you explain to me something? i_m looking here https://github.com/D-Programming-Language/druntime/blob/v2.066.1/src/rt/lifetime.d#L591 --- line #603 auto size = ti.next.tsize; why `next`? it can be even null if this is last TypeInfo in the linked list. - btw, i used suggested trackallocs.d and GC defenetely receives NO_SCAN before tag: 1 len: 2 ptr: 103A78058 root: 103A77000:8192 attr: APPENDABLE gc_qalloc(41, NO_SCAN APPENDABLE ) cc: 29106 asz: 10152603, ti: null ret: BlkInfo_(104423800, 64, 10) after tag: 1 len: 3 ptr: 104423810 root: 104423800:64 attr: NO_SCAN APPENDABLE
Re: Garbage collector collects live objects
On Friday, 12 December 2014 at 12:53:00 UTC, Ruslan Mullakhmetov wrote: On Thursday, 11 December 2014 at 18:36:59 UTC, Steven Schveighoffer wrote: My analysis so far: 4. If your code is multi-threaded, but using __gshared, it can make the cache incorrect. Are you doing this? the app is multi-threaded via std.concurrency. there is only one known to me place where __gshared is used: logging library (checked by searching through whole source tree). make stub for this lib and try, so identify whether cache invalidated by _gshared or not. removing __gshared seems does not helped.
Re: Garbage collector collects live objects
On Friday, 12 December 2014 at 15:50:26 UTC, Steven Schveighoffer wrote: Can I email you at this address? If not, email me at the address from my post to let me know your contact, no reason to work through building issues on the public forum :) -Steve reach me at theambient [] me__com
Re: Garbage collector collects live objects
On Wednesday, 10 December 2014 at 02:43:19 UTC, ketmar via Digitalmars-d-learn wrote: On Tue, 09 Dec 2014 17:18:44 + Ruslan Mullakhmetov via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: but i still have no clue how to overcome GC =( why do you want to fight with GC? most of the time GC is your friend. see the topic: i got corruption when dereferencing object.
Re: Garbage collector collects live objects
On Tuesday, 9 December 2014 at 21:38:57 UTC, Steven Schveighoffer wrote: On 12/9/14 2:56 PM, Steven Schveighoffer wrote: On 12/9/14 12:40 PM, Ruslan Mullakhmetov wrote: array holds 11 64bit pointers but it's block size is only 128 bytes 11 * 64 = 704 bytes. what's wrong with this arithmetics? Hah, just realized what's wrong. It's not 64 *bytes* per pointer, it's 64 *bits*. So 8 bytes. 11 * 8 == 88. Starting to sound more and more normal... -Steve yes. that was the mistake. also after fixing bug in Blk Attributes printing i got more reasonable attrs for object blk: FINALIZE for array of objects blk: NO_SCAN APPENDABLE this is sound good except for NO_SCAN. I did simple test file in which allocate array of Foo objects (http://dpaste.dzfl.pl/89ab00a897f6) there i see blk attrs only APPENDABLE without NO_SCAN. as far as i understand GC will not scan this array for references and those if the only reference to object is stored in this array will not see it, those assume this object as **not** referenced and collects it, am i right? the other question why this happens... try to debug more.
Re: Garbage collector collects live objects
On Wednesday, 10 December 2014 at 08:46:12 UTC, Ruslan Mullakhmetov wrote: yes. that was the mistake. also after fixing bug in Blk Attributes printing i got more reasonable attrs for object blk: FINALIZE for array of objects blk: NO_SCAN APPENDABLE this is sound good except for NO_SCAN. ... the other question why this happens... try to debug more. I've done more dubugging. what i've found: initially array blk has only attrs APPENDABLE, but after some time this blk is shrinked and reallocated (moved) and then NO_SCAN attr appears. here the output of my extended logs: before tag: 1 len: 2 ptr: 103DD9058 root: 103DD8000:8192 attr: APPENDABLE after tag: 1 len: 3 ptr: 103A21DD0 root: 103A21DC0:64 attr: NO_SCAN APPENDABLE this is produced by the following code http://dpaste.dzfl.pl/0c6dc16270a1 so in a nutshell after appending to array via ~= operator blk attrs changed from APPENDABLE to NO_SCAN APPENDABLE which cause the problem. why and how this happens? can anybody explain it to me?
Re: Garbage collector collects live objects
On Wednesday, 10 December 2014 at 12:52:24 UTC, Ruslan Mullakhmetov wrote: why and how this happens? can anybody explain it to me? I tried to extract this and saw NO NO_SCAN attrs after moving blk: the following piece of output produced by http://dpaste.dzfl.pl/6f773e17de92 len: 6 ptr: 109DF0010 root: 109DF:1048576 attr: APPENDABLE len: 7 ptr: 109DF0010 root: 109DF:1048576 attr: APPENDABLE len: 8 ptr: 109DF0010 root: 109DF:1048576 attr: APPENDABLE len: 9 ptr: 109DF0010 root: 109DF:1048576 attr: APPENDABLE len: 10 ptr: 109DF0010 root: 109DF:1048576 attr: APPENDABLE --- shrinked -- len: 1 ptr: 109EB3508 root: 109DF:1048576 attr: APPENDABLE len: 2 ptr: 109EB3508 root: 109DF:1048576 attr: APPENDABLE len: 3 ptr: 109EB3508 root: 109DF:1048576 attr: APPENDABLE len: 4 ptr: 109EB3508 root: 109DF:1048576 attr: APPENDABLE len: 5 ptr: 109EB3508 root: 109DF:1048576 attr: APPENDABLE len: 6 ptr: 109F60640 root: 109F60640:64 attr: APPENDABLE len: 7 ptr: 109F60640 root: 109F60640:64 attr: APPENDABLE
Re: Garbage collector collects live objects
On Wednesday, 10 December 2014 at 13:00:45 UTC, ketmar via Digitalmars-d-learn wrote: can you give us a minified code that causes this behavior? see previous post. the problem vanish if i try to extract it.
Garbage collector collects live objects
Hi, I experience very strange problem: GC somehow collects live objects. I found it because i got segfaults. After debugging and tracing i found this is because of accessing not allocated memory. I did the following checks: - added to some class invariant check for access to suspicious members with assertion assert(GC.addrOf(cast(void*)x) !is null); where it fails DETERMINISTICALLY at some point - printing address of allocated classes where i observe the following pattern - ctor check check check - dtor check (fails) could anybody advice me with something? I got really frustrated by this strange behaviour which i can not fix right now. key observations: - it is deterministically behaviour (what gets me even more confused cause GC collections as far as i know runs from time to time) - i do not play with pointers optimisation like hiding its in ints or floats. - i operate with large uniformly distributed (video) data in memory where pointer like patterns may occur. but this is not the case cause (1) it brings at worst long living objects (2) input sequence constant but allocated pointers each run different.
Re: Garbage collector collects live objects
On Tuesday, 9 December 2014 at 14:23:06 UTC, Steven Schveighoffer
wrote:
On 12/9/14 8:54 AM, Ruslan Mullakhmetov wrote:
Hi,
I experience very strange problem: GC somehow collects live
objects.
I found it because i got segfaults. After debugging and
tracing i found
this is because of accessing not allocated memory.
I did the following checks:
- added to some class invariant check for access to suspicious
members
with assertion
assert(GC.addrOf(cast(void*)x) !is null);
where it fails DETERMINISTICALLY at some point
- printing address of allocated classes where i observe the
following
pattern
- ctor
check
check
check
- dtor
check (fails)
could anybody advice me with something? I got really
frustrated by this
strange behaviour which i can not fix right now.
key observations:
- it is deterministically behaviour (what gets me even more
confused
cause GC collections as far as i know runs from time to time)
- i do not play with pointers optimisation like hiding its in
ints or
floats.
- i operate with large uniformly distributed (video) data in
memory
where pointer like patterns may occur. but this is not the
case cause
(1) it brings at worst long living objects (2) input sequence
constant
but allocated pointers each run different.
A random guess, since you haven't posted any code, are you
accessing GC resources inside a destructor? If so, that is not
guaranteed to work. A class destructor, or a destructor of a
struct that is contained inside a class, can only be used to
destroy NON-GC resources.
If you want more help, you need to post some code. Something
that minimally causes the issue would be good.
-Steve
No, there is no accessing GC resources in dtors.
the only usage of dtor in one class is
~this()
{
_file.close();
}
where _file is of type std.file.File
i'll try to extract problem to any observable source code but all
my previous attempts lead to problem being diminish.
Re: Garbage collector collects live objects
On Tuesday, 9 December 2014 at 16:53:02 UTC, Steven Schveighoffer
wrote:
On 12/9/14 11:17 AM, ketmar via Digitalmars-d-learn wrote:
On Tue, 09 Dec 2014 14:52:53 +
Ruslan Mullakhmetov via Digitalmars-d-learn
digitalmars-d-learn@puremagic.com wrote:
On Tuesday, 9 December 2014 at 14:23:06 UTC, Steven
Schveighoffer
wrote:
On 12/9/14 8:54 AM, Ruslan Mullakhmetov wrote:
Hi,
I experience very strange problem: GC somehow collects live
objects.
I found it because i got segfaults. After debugging and
tracing i found
this is because of accessing not allocated memory.
I did the following checks:
- added to some class invariant check for access to
suspicious
members
with assertion
assert(GC.addrOf(cast(void*)x) !is null);
where it fails DETERMINISTICALLY at some point
- printing address of allocated classes where i observe the
following
pattern
- ctor
check
check
check
- dtor
check (fails)
could anybody advice me with something? I got really
frustrated by this
strange behaviour which i can not fix right now.
key observations:
- it is deterministically behaviour (what gets me even more
confused
cause GC collections as far as i know runs from time to
time)
- i do not play with pointers optimisation like hiding its
in
ints or
floats.
- i operate with large uniformly distributed (video) data in
memory
where pointer like patterns may occur. but this is not the
case cause
(1) it brings at worst long living objects (2) input
sequence
constant
but allocated pointers each run different.
A random guess, since you haven't posted any code, are you
accessing GC resources inside a destructor? If so, that is
not
guaranteed to work. A class destructor, or a destructor of a
struct that is contained inside a class, can only be used to
destroy NON-GC resources.
If you want more help, you need to post some code. Something
that minimally causes the issue would be good.
-Steve
No, there is no accessing GC resources in dtors.
the only usage of dtor in one class is
~this()
{
_file.close();
}
where _file is of type std.file.File
i'll try to extract problem to any observable source code but
all
my previous attempts lead to problem being diminish.
that file can be already finalized. please remember that
`~this()` is
more a finalizer than destructor, and it's called on *dead*
object.
here this means that any other object in your object (including
structs) can be already finalized at the time GC decides to
call your
finalizer.
File is specially designed (although it's not perfect) to be
able to close in the GC. Its ref-counted payload is placed on
the C heap to allow access during finalization.
That being said, you actually don't need to write the above in
the class finalizer, _file's destructor will automatically be
called.
just avoid destructors unless you *really* need that. in
your case
simply let GC finalize your File, don't try to help GC. this
is not C++
(or any other language without GC) and destructors aren't
destructing
anything at all. destructors must clean up the things that
GC cannot
(malloc()'ed memory, for example), and nothing else.
Good advice ;)
I would say other than library writers, nobody should ever
write a class dtor.
-Steve
thanks, I got it: either C++ or D dtors are minefield =)
but i still have no clue how to overcome GC =(
Re: Garbage collector collects live objects
On Tuesday, 9 December 2014 at 16:13:25 UTC, Dicebot wrote: It may happen if only reference to an object is stored in memory block marked as data-only (using ubyte[] for a buffer is probably most common reason I have encountered) Thanks for interesting hypothesis, but that's not the issue. innocent though collected objects are living in D array MyClass[] which are living in assoc array as value. i checked attributes for GC block holding this array: ``` FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR ``` I really doubting about NO_INTERIOR. can anybody confirm me that is's working with array slicing which i heavily use? also i found that block size is quite small pre array: [100A2FD00, 100A2F700, 100A33B80, 100A33500, 100A3FE80, 100A3F980, 100A3F400, 100A72600, 100A7DF80, 100A7DA80, 100A7D500] array ptr: 100A72580 root: 100A72580:128 attr: FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR [100985A00] keys: [1] as: 1 au: 100A2FD00 [100985A00] keys: [1] as: 1 au: 100A2F700 [100985A00] keys: [1] as: 1 au: 100A33B80 /pre array holds 11 64bit pointers but it's block size is only 128 bytes 11 * 64 = 704 bytes. what's wrong with this arithmetics?
Re: Garbage collector collects live objects
On Tuesday, 9 December 2014 at 19:56:30 UTC, Steven Schveighoffer wrote: On 12/9/14 12:40 PM, Ruslan Mullakhmetov wrote: On Tuesday, 9 December 2014 at 16:13:25 UTC, Dicebot wrote: i checked attributes for GC block holding this array: FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR That does not sound right at all. No block should ever have both FINALIZE (reserved for objects only) and APPENDABLE (reserved for arrays only). also i found that block size is quite small pre array: [100A2FD00, 100A2F700, 100A33B80, 100A33500, 100A3FE80, 100A3F980, 100A3F400, 100A72600, 100A7DF80, 100A7DA80, 100A7D500] array ptr: 100A72580 root: 100A72580:128 attr: FINALIZE NO_SCAN NO_MOVE APPENDABLE NO_INTERIOR [100985A00] keys: [1] as: 1 au: 100A2FD00 [100985A00] keys: [1] as: 1 au: 100A2F700 [100985A00] keys: [1] as: 1 au: 100A33B80 /pre array holds 11 64bit pointers but it's block size is only 128 bytes 11 * 64 = 704 bytes. what's wrong with this arithmetics? I think there is something you are missing, or something is very corrupt. Can you show the code that prints this? -Steve here the piece of code i used to output this value http://pastebin.com/cQf9Nghp StreamIndex is ubyte AccessUnit is some class
Re: trace GC work
On Tuesday, 18 February 2014 at 22:59:00 UTC, Mike wrote:
On Tuesday, 18 February 2014 at 18:10:40 UTC, Ruslan
Mullakhmetov wrote:
Hi,
Is it possible to trace GC allocation calls to determine times
of program death and have some stats?
So i want the following information:
- garbage collection starts at {time}
- garbage collection stops at {time}
- (optionally) gc took {time}, collected {size} bytes
thanks.
I believe you'll have to modify the runtime to achieve this.
See the gc folder[1]. An example of a custom garbage
collector is in the gcstub folder[2].
Mike
[1]
https://github.com/D-Programming-Language/druntime/tree/e47a00bff935c3f079bb567a6ec97663ba384487/src/gc
[2]
https://github.com/D-Programming-Language/druntime/blob/e47a00bff935c3f079bb567a6ec97663ba384487/src/gcstub/gc.d
thanks, but i hoped it would be more graceful.
trace GC work
Hi,
Is it possible to trace GC allocation calls to determine times of
program death and have some stats?
So i want the following information:
- garbage collection starts at {time}
- garbage collection stops at {time}
- (optionally) gc took {time}, collected {size} bytes
thanks.
std.bigint.BigInt and const modifier
BigInt is a struct == ValueType, suppose i can assign
const(BigInt) to BigInt
const(BigInt)/const(BigInt) do not compile.
Is it a bug or design considerations?
test.d
import std.stdio;
import std.bigint;
void foo(BigInt b)
{
std.stdio.writefln(foo(%s), b);
}
void main()
{
const BigInt b = BigInt(445216415446);
auto b1 = b / b;
foo(b);
}
$ dmd test.d
/tmp/test.d(14): Error: 'b' is not of arithmetic type, it is a
const(BigInt)
/tmp/test.d(14): Error: 'b' is not of arithmetic type, it is a
const(BigInt)
/tmp/test.d(16): Error: function test.foo (BigInt b) is not
callable using argument types (const(BigInt))
Re: std.bigint.BigInt and const modifier
On Sunday, 16 February 2014 at 20:36:32 UTC, bearophile wrote: Ruslan Mullakhmetov: Is it a bug or design considerations? It was a const correctness bug, but it's already fixed in recent D compilers. Bye, bearophile recent D compilers. i use v2.064.2. is it outdated ?
Re: core.sys.posix.unistd link error
Didn't catch. How can I use it at runtime? I can not link to actually C function? On Saturday, 21 September 2013 at 19:40:48 UTC, Jonathan M Davis wrote: On Saturday, September 21, 2013 20:30:00 Ruslan Mullakhmetov wrote: i use pipe() syscall from my program. when i compile it I got the following msg: Error: pipe cannot be interpreted at compile time, because it has no available source code how can i fix it? dmd 2.063.2, Mac OS X It sounds like you're trying to use pipe at compile time, and as the error says, you can't use it at compile time, because no source code for it is available. The same goes for all C functions. They can only be used at runtime. - Jonathan M Davis
Re: core.sys.posix.unistd link error
I found where the problem is.
I used a system call (external C function) in class ctor. then I
declared global variable of this class and INITIALZIED that
variable inplace. If i move initalization in module static this()
everything compiles.
the code is:
incorrect version:
http://dpaste.com/hold/1391530/
correct:
http://dpaste.com/hold/1391523/
But now i need to sort out what the difference between
// global scope
int a = 10;
and
int a;
static this()
{
a = 10;
}
I appreciate if somebody give a link or chapter number where to
read.
Re: core.sys.posix.unistd link error
I would be curious to see why you believe them to be the same.
Cause i'm a C++ programmer and there is no such thing as module
and module initializer, in fact object file initialization
consist of initialization of all its static variables somewhen
before the first call of a function in that object file (if any).
On Sunday, 22 September 2013 at 19:50:14 UTC, Dmitry Olshansky
wrote:
22-Sep-2013 15:52, Ruslan Mullakhmetov пишет:
I found where the problem is.
I used a system call (external C function) in class ctor. then
I
declared global variable of this class and INITIALZIED that
variable
inplace. If i move initalization in module static this()
everything
compiles.
the code is:
incorrect version:
http://dpaste.com/hold/1391530/
correct:
http://dpaste.com/hold/1391523/
But now i need to sort out what the difference between
// global scope
int a = 10;
This just puts calculated value 10 into TLS data section as
initializer for a.
and
int a;
static this()
{
a = 10;
}
This defines a global with 0 initializer.
Then static this is a function that is executed for each D
thread on creation, following the module dependency chain (i.e.
if there is static this in imported module it should be run
first).
I would be curious to see why you believe them to be the same.
I appreciate if somebody give a link or chapter number where
to read.
Re: core.sys.posix.unistd link error
Thanks. I suspected it but i wanted a formal reference. the
logic, though little bit cleared by you is quite obvious. But
don't waste time, if you can not tell from a scratch that this
is clause x.y.z of the Standard, sorry, Book.
On Sunday, 22 September 2013 at 19:56:36 UTC, Jonathan M Davis
wrote:
On Sunday, September 22, 2013 13:52:54 Ruslan Mullakhmetov
wrote:
But now i need to sort out what the difference between
// global scope
int a = 10;
That directly initializes the variable at compile time, meaning
that whatever
is used to initialize the variable must be callable at compile
time. And the
value must be able to be set at compile time and then be
carried over to
runtime. That will work with int, but it does not work with
most stuff that's
on the heap (like classes or AAs) - arrays would be the major
exception to
that, since they can be set at compile time (and I believe that
it was
recently changed so that immutable classes could be set at
compile time, but
not const or mutable ones - implementing that is rather
complicated, and it
may or may not ever happen). Over time, what you can do at
compile time with
CTFE (Compile Time Function Evaluation) has improved, but there
are still
restrictions, and some things will never be possible (e.g. I/O
or calling C
functions).
and
int a;
static this()
{
a = 10;
}
That does not set the variable at compile time. Rather, the
static constructor
sets it at runtime. So, this has none of the restrictions that
directly
initializing a module or static variable does. However, it does
have the
downside that two modules that have static constructors can't
import each
other (either directly or indirectly), because then the runtime
wouldn't know
which order to run them in. If you do that, you'll get an
exception at runtime
complaining about a circular import (which sucks, but
unfortunately, the
circular import can't always be detected at compile time -
thanks in part to
.di files - so runtime detection is the best that can be done).
So, while
static constructors can be really nice, you do have to avoid
having modules
that use them import each other, which means either being
careful about how
your modules import each other or avoiding static constructors.
Which is
easier depends on your code.
core.sys.posix.unistd link error
i use pipe() syscall from my program. when i compile it I got the following msg: Error: pipe cannot be interpreted at compile time, because it has no available source code how can i fix it? dmd 2.063.2, Mac OS X
std.concurrency.receive() question
i try to compile the following slightly modified sample from the
book and it failed with the following messages
=== source
import std.stdio;
import std.concurrency;
void fileWriter()
{
// Write loop
for (bool running = true; running; )
{
receive(
(immutable(ubyte)[] buffer) {},
(OwnerTerminated) { running = false; }
);
}
stderr.writeln(Normally terminated.);
}
void main()
{
}
== error messages ===
/Users/ruslan/Source/dlang/dub-test/source/listener.d(11): Error:
template std.concurrency.receive does not match any function
template declaration. Candidates are:
/usr/local/Cellar/dmd/2.063.2/libexec/src/phobos/std/concurrency.d(646):
std.concurrency.receive(T...)(T ops)
/Users/ruslan/Source/dlang/dub-test/source/listener.d(11): Error:
template std.concurrency.receive(T...)(T ops) cannot deduce
template function from argument types !()(void
function(immutable(ubyte)[] buffer) pure nothrow @safe, void)
what's wrong? if i replace OwnerTerminated with int or simply
remove everything is ok.
if i replace with my own struct Terminate - fail.
any help would be appreciated.
Re: working on the dlang.org website
On Sunday, 14 July 2013 at 04:56:46 UTC, Val Markovic wrote: So, Andrei also mentioned how the community needs to improve the dlang.orgsite and make it a first-class priority. It's the point of contact for newcomers so it's very important we get it right. Since I know a thing-or-two about web development, I decided to look into improving the site. Let me put my two cents at this thread, hope it would be useful. (1) The thread went (from the start) to discussion of technical details, not content discussion, and users are interested in content itself, which, IMHO is not optimally organized. (2) Flashing and slow responses are annoying but not so crucial (as from my experience), thought somebody waits for up to 7 seconds to load web page - agree, that should be fixed, but again, i do not have such problem or didn't noticed (why?). (3) dlang.org content organisation: (3.1) start page describes the language - very good, but it would be better if it also contains clear links (the big big buttons) to overview, download page, tutorial, documentation; (3.2) as were already mentioned the documentation should be extracted to dedicated site (either subdomain or subfolder) mainly to remove unnecessary at this section main site navigation. technically it would be better because documentation for sure have different markup / layout / page structure and probably would have different presentation. As for me I loved http://www.erlang.org/doc/man/edoc.html - clear, navigable, lightweight; (3.3) explicit language tutorial section would be great benefit (not doc-book-tutorial), language tour like for go language would be excellent. (3.4) explicit getting started page (infrastructure preparing) (3.4) are articles part of documentation? (4) design and appearance: (4.1) fixed width layout - more readable, all typesetting books recommends line to be no more then 66 characters long; (4.2) scrolling navigation pane is annoying. (4.3) more lightweight and readable design: lighter background tones and darker foreground tones (people get used to read black symbols on white paper), thiner or zero borders, more explicit headers. (5) external and outdated content: there are different d specific content outside dlang.org (i.e. source.org) containing tutorials, articles, source codes that is outdated and doesn't even compile. this mislead newbie, gets him irritated and force to quit learning language. if it is possible this content should be revised and incorporated to dlang.org and removed, because it breaks down all language marketing. This just my thought that i hope could be useful, cause i'm a visual person type and got easily embarrassed by a lot of content especially when it is not well-formed =)
std.concurrency.receive() and event demultiplexer
Hi, I'm playing around with std.concurrency and found it quite interesting. I drop all praise words (they all already said) and go to the question itself. With std.concurrency we could have a number of asynchronously operating routines doing job linearly, for instance, reading from socket and sending received data to consumer thread. that ok. but what if socket or generally handling task blocks indefinitely? We lose ability to respond to external world (other threads). Okay, we could employ async event-based model for handling our task (reading socket), but now we would block in event demultiplexer loop. Now we could go further and overcome it with complication of logic: - timed wait for event on socket - when timeout occur check receive for incoming messages again with timeout - switch back to events waiting The drawbacks are obvious: - complicated logic - artificial switching between two demultiplexers: event loop and std.concurrency.receive() - need to choose good timeout to meet both: responsiveness and cpu load Alternatively it is possible to take away all blocking operations to another child thread, but this does not eliminate problem of resource freeing. with socket example this is dangerous with hanging socket descriptor and (1) not telling peer socket to shut up conversation (2) overflow of number of open file descriptors The problem would be solved quite elegant if either (1) receive() could handle different events, not just communication messages, (2) it would be possible to get waitable descriptor for receive() that could be registered in 3-party event demultiplexer. Of course receive() is not aimed for (1) and i know no ways to get (2) working. So the question: how to overcome the problem i described if i described it clear.
Re: Is there an equivalent of Rust owned pointer in D?
On Tuesday, 6 August 2013 at 16:32:43 UTC, Nick Treleaven wrote: On 06/08/2013 06:26, Elvis wrote: Owned pointer in Rust is a good design IMO, how about D equivalent? I think this is about the closest ATM: http://dlang.org/phobos/std_typecons.html#.Unique would be good to have it as language feature like scoped reference. this clear code (as from c++11 experience abundant std::unique_ptrT get me irritated). And of course unique idiom (ownership) get D usable for realtime (removed GC)
Re: Is there an equivalent of Rust owned pointer in D?
On Friday, 9 August 2013 at 19:05:26 UTC, Tobias Pankrath wrote: On Friday, 9 August 2013 at 18:16:57 UTC, Ruslan Mullakhmetov wrote: On Tuesday, 6 August 2013 at 16:32:43 UTC, Nick Treleaven wrote: On 06/08/2013 06:26, Elvis wrote: Owned pointer in Rust is a good design IMO, how about D equivalent? I think this is about the closest ATM: http://dlang.org/phobos/std_typecons.html#.Unique would be good to have it as language feature like scoped reference. this clear code (as from c++11 experience abundant std::unique_ptrT get me irritated). And of course unique idiom (ownership) get D usable for realtime (removed GC) While I'm convinced that unique pointers / types are a important concept, especially if combined with immutable. (What's transitive unique/immutable can be cast to immutable .. ), I don't think that just adding a new type modifier will lead D anywhere. No, I didn't assert that. I just proposed a synthetic sugar =). Unique pointer/types are already in D (at the library level).
Re: d future or plans for d3
On 2011-12-19 11:52:25 +, Alex Rønne Petersen said: On 18-12-2011 15:40, Somedude wrote: Le 18/12/2011 15:07, Ruslan Mullakhmetov a écrit : GC is just a mater of implementation. In presence of resources implement good GC algorithm offers no difficulty. Oh really ? Then please make us a favor and write one for D. Also I'm sure the C++ guys will be pleased to hear that it's such an easy task. Yeah, unfortunately, we can't just keep saying it's an implementation issue. It's very much a real problem; D programmers are *avoiding* the GC because it just isn't comparable to, say, the .NET and Java GCs in performance at all. - Alex Thanks for you explanation. I'm quite far away from GC but where is the problem compare to Java and .NET? Resources (people), specific language features making hard to implement GC or something else? When i said that this is just a matter of implementation i followed the idea that it's already implemented in say Java, C#, Erlang wich GC was declared to be good enough in this topic. -- BR, Ruslan Mullakhmetov
Re: d future or plans for d3
On 2011-12-18 00:56:33 +, Timon Gehr said: C++11 does not change the relation between D and C++ a lot. Why do you think it does? Because it incorporates many features D declared to be unique to it over C++ like - thread local variables - explicit concurrency model - type deduction - variadic templates - generalized constant expressions Some comparison is made at http://d-programming-language.org/cpp0x.html The language does not have to be changed to get that to work. The C language doesn't have to be changed to get OOP working. There are libraries written in plain C satisfying all requirements of OOP, e.g. libav. Nevertheless C++ was born. So, what do i propose. To get it explicit in language and working out of the box, like in Erlang. the only benefit over Erlang i currently see that D is much more friendly for newcomers from C-like camp. The another is possibility for embedded programming. I was surprised that there are attempts to use MAS at embedded programming e.g. robotics where different controlers are autonomous and communicate with each others. comparing to other modern languages IMO that is not a very important question. It is not a contest. I thin that this is exactly context. I try to explain. D has reputation of marginal language with no concrete niche. For embedded programming C++ and rather plain C is used. For high level programming Java/C# are good enough. For distributed programming - Erlang. For scientific - Matlab, python, fortran, C. When i tell somebody about D the first question i get except community, maturity and stability is where it can be used. Where it gives true advantage. And i can not answer. So i propose to introduce new paradigm, despite of it is _not_ technical problem. it like ajax where combination of parts gave result greater then sum of parts. One may consider it as marketing trick to attract attention to D. On 12/18/2011 01:09 AM, Ruslan Mullakhmetov wrote: Hi all, I want to ask you about D future, i mean next big iteration of D and propose some new feature, agent-based programming. Currently, after introducing C++11 i see the only advantages of D over C++11 except syntax sugare is garbage collector and modules. C++11 does not change the relation between D and C++ a lot. Why do you think it does? I recentrly attended student school (workshop) on multi-agent systems (MAS)and self-organizing sysetems. I was really impressed and thought that this probably is the silver bullet which Brucks declared to be absent. I mean agent-based programming as foundation of self-organzing systems. If you are interested you can find a lot of information by googling. So I would like to get your feedback to introduce new paradigm, paradigm of agent programming into D. Actually, I'm not deep into MAS, but as far as i know it's just autonomous class, i.e. class that has it's own independent context of execution that can communicate with other parties (agents) and can affect on environment if any (like ant). So it would be nice to have this in language core/library. There is erlang that already satisfied all requirements (as far as i know) of MAS language. So the question is does D need to take this paradigm? - Or concentrate on its current paradigms? The only advatnage ovder erlang i see is that D propose itself as embedded programming language which erlang do not satisfy (am i right?). So i need your feedback on the following: (i) do you think that D needs to adsorb agent-programming paradigm The language does not have to be changed to get that to work. (ii) can it benefit D Yes. comparing to other modern languages IMO that is not a very important question. It is not a contest. -- BR, Ruslan Mullakhmetov
Re: d future or plans for d3
I do not want to make a flame over D vs C++11. I answered else thread on some differences of C++11 vs D, but i agree with you that i was a little bit in a hurry. After rechecking i remembered constraints, static compilation (static if), contract programming, functional programming. perhaps i skipped something. For those features you mentioned: - templates are know quite sane, but D still outperforms it. The question is how many people would benefit from this difference. - thread local variables: thread local storage (though not default) - message passing implemented via libraries: boost.task? - immutables are only needed for functional programming which in turn necessary for parallelizing / distributed applications / formal verification. I don't know, does D (compiler) utilize it? Once again, i'm not trying to make a holywar. I'm D lover myself. But a lot of people do not consider this benefits of D enough to shift to it as for my opinion and experience. On 2011-12-18 01:53:14 +, Timon Gehr said: On 12/18/2011 02:42 AM, a wrote: Ruslan Mullakhmetov Wrote: Currently, after introducing C++11 i see the only advantages of D over C++11 except syntax sugare is garbage collector and modules. So you are saying that sane templates, range based standard library and concurrency improvements (thread local variables, immutable, message passing) are all just syntactic sugar? And you didn't even mention CTFE and code generation yet. -- BR, Ruslan Mullakhmetov
Re: d future or plans for d3
It will be years before we seriously start looking at D3 Absolutely. Just wanted to share with my thoughts. When i wrote to NG i do not intend to say Let do it right know, instead wanted to say When D3 would be considered... Any good movement as i hope D is ought to have some global view and global goals. I think, new paradigm would benefit D. Also as already mentioned else thread this can be used like marketing trick. On 2011-12-18 02:26:51 +, Jonathan M Davis said: On Sunday, December 18, 2011 04:09:21 Ruslan Mullakhmetov wrote: I want to ask you about D future It will be years before we seriously start looking at D3, and while there are ideas for what we might like to do with it, it's far too early to say what's likely to happen with it. D2 needs to be fully stabilized and be in general use for a while before we really look at expanding it into D3. We really need to work at making D2 a success before we worry about where we're going next. Also, many of the best changes to D3 won't be evident until D2 has been used enough for its problems to become evident. D1 and D2 manage to improve on C++'s problems as well as they do precisely because we know what they are. We don't really know what D2's primary problems are yet, and that will take time. Also, while D3 may be years off, it could be that after D2 has stabilized more, we'll add new features that are backwards compatible. So, just because D3 is years off does not necessarily mean that D2 is static. In addition, many things can be done in libraries without needing to add anything to the language, so what you can do with D2 will continue to improve, even if the language itself doesn't change much. Regardless, the main focus right now is in stabilizing dmd and fleshing out the standard library, not in creating a new version of D with new features. It's too early for that. - Jonathan M Davis -- BR, Ruslan Mullakhmetov
Re: d future or plans for d3
As far as I understand Erlang performance is reached by horizontal scheduling (i'm not sure i used exact word) by which i mean that if you have lack of performance you can buy new hardware nodes (servers) and without any changes in code and even recompiling you can distribute workload. That is very good approach in situation when processor stopped getting faster but become more parallelized and cheaper. GC is just a mater of implementation. In presence of resources implement good GC algorithm offers no difficulty. All languages have their place sure. and many people doubt of place of D. It would be nice if such good lang will find it's place. On 2011-12-18 12:19:43 +, Alex Rønne Petersen said: On 18-12-2011 12:45, Somedude wrote: Le 18/12/2011 12:13, Ruslan Mullakhmetov a écrit : I do not want to make a flame over D vs C++11. ... Once again, i'm not trying to make a holywar. I'm D lover myself. But a lot of people do not consider this benefits of D enough to shift to it as for my opinion and experience. These people will not change their mind whatever you throw at them. For them, it's a matter of religion, not a matter of comparison. We shouldn't bother pleasing such or such group of people. BTW, your comparison with Erlang misses one crucial point: performance. A part for some very specific applications for which it is designed, Erlang's general performance is simply not comparable to that of D. A lot of D's complex set of features is geared towards increasing runtime performance. Erlang just doesn't compare. On the other hand, Erlang's runtime is built around message-passing. I *extremely strongly doubt* that you can get similar performance out of message-passing in D, for two reasons: 1) The Erlang runtime is written in highly optimized C. 2) Erlang uses better garbage collection strategies than D. On point 1: Of course, you can write a message-passing implementation in D that's basically just glorified C, but that defeats the point of using D in the first place IMHO. On point 2: This is highly unlikely to change. It seems (from reading the NG's past discussions) that there is virtually no interest from the core devs of D to do The Right Thing to make precise garbage collection possible, sadly (and neither to make thread-local GC possible, and other such techniques (Erlang runs a separate GC per Erlang process, very successfully)). Yes, Erlang's emulator is not good in raw performance. It was never geared for this (that's why it has easy interfacing with C for performance-sensitive code). However, things like HiPE improve the situation. All languages have their place, and Erlang is still going strong in the world of massive concurrency. I doubt D will be able to challenge this until its GC issues have been resolved (and even then, D is nowhere near as convenient, lacking fault tolerance and location transparency; sometimes these traits are *much* more desirable than raw performance when dealing with concurrency on such high levels). - Alex -- BR, Ruslan Mullakhmetov
Re: d future or plans for d3
I think there is no need in language changes. Everythink can be implemented via library. What is needed: - base agent class - base behaviors - runtime that provide ability to run independent agents even in single-thread mode. Number of agents can be greater then number of treads. so scheduler is needed. - communication platform to delver messages between agents with possibility of broadcasting, multicasting, unicasting (normal). - service discovery mechanism allowing to register/deregister agents and query for agents with specific roles. I will prepare more detail answer to the weekend when i get free. As for applications, i see embedded market: communicating with each other phones via bluetooth/wi-fi/GSM/CDMA, robotics and others. At least it my hopes. On 2011-12-18 22:23:28 +, Andrei Alexandrescu said: On 12/18/11 3:07 PM, Timon Gehr wrote: On 12/18/2011 11:51 AM, Ruslan Mullakhmetov wrote: On 2011-12-18 00:56:33 +, Timon Gehr said: C++11 does not change the relation between D and C++ a lot. Why do you think it does? Because it incorporates many features D declared to be unique to it It does not, except for the most trivial stuff. I think it's best to not derail the argument into a D vs. C++ thing. Ruslan, what language and what library features do you envision should be added, and what would be their most likely applications? Thanks, Andrei -- BR, Ruslan Mullakhmetov
Re: .dmg installer for OSX?
On 2011-12-11 22:10:37 +, Peter Alexander said: On 11/12/11 9:29 PM, Andrei Alexandrescu wrote: There's something missing from the download page (to which I just added some nice logos): http://www.d-programming-language.org/download.html That would be an OSX .dmg. Is a member of the community interested in taking up such a project? Thanks, Andrei A .dmg isn't an installer. It's just an archive containing files. Usually it contains an application package that the window background instructs you to drag into the Applications folder... but that isn't applicable to DMD. Perhaps a .pkg would be more appropriate? http://en.wikipedia.org/wiki/.dmg http://en.wikipedia.org/wiki/.pkg Probably, Mac App Store instalation would be the best? I don't know, do Apple policies allow to install a command line utilities via Macc App Store, but at least apple itself install OS X Lion and XCode via MAS. So, there is technical posibility to install arbitary toolset, not only pure GUI.app located in /Application. What do you say? -- BR, Ruslan Mullakhmetov
Re: State of LDC
On 2011-11-12 11:29:51 +, Jonas Drewsen said: dlang.org actually contains a copy of d-p-l.org. Maybe the owner of dlang.org is willing to let digitalmars take over that domain and make it the official domain. That would be much better that d-p-l.org I think. Wow! I didn't know. that's good. -- BR, Ruslan Mullakhmetov
Re: State of LDC
On 2011-11-12 00:06:27 +0400, David Nadlinger said: When we started the move to GitHub, I tried to create an »ldc« org, but it was already taken. The best alternative we could come up with on IRC was ldc-developers, which we then decided to use. Regarding d-programming-ldc, I am not much sure if it would really change anything, but if it is generally agreed on, fine with me. Not having formally announced the move could actually come in handy here, since the only thing that depends on the path (besides quite a number of local user repos) is probably the Fedora packaging script. what about ldc.llvm.org? I think it's possible. as for d-p-l.org, it seems to me to be too long despite of it's currently the in first place at chrome start page ))) i looked up fo dpl.* but it apperntly taken. Probably this topic has been already discussed. -- BR, Ruslan Mullakhmetov
pass array of objects to spawn
Hi folks,
I need to create thread and pass to it array of objects wich will no
longer use in main thread.
The problem is that spawn only accepts immutables and i have no idea
how to tell him that i transfer full ownership of the data to child
thread.
the code:
import std.concurrency;
class Foo
{
}
void worker( Foo[] data )
{
//...
}
void main()
{
auto data = new Foo[10];
spawn( worker, data );
}
P.S. I do need to create data in main thread and distribute it to child
threads.
--
BR, Ruslan Mullakhmetov
Re: pass array of objects to spawn
On 2011-11-11 01:21:09 +0400, Ali Çehreli said:
class Foo
{
}
void worker( shared(Foo)[] data )
{
//...
}
void main()
{
auto data = new shared(Foo)[10];
spawn( worker, data );
}
Thanks. I tried to use the second version, a lttle bit modified it for
actual mutation and it is failed to compile with error
thread.d(17): Error: function thread.Foo.mutate () is not callable
using argument types () shared
the code:
import std.exception;
import std.concurrency;
class Foo
{
public int val;
void mutate()
{
val = 1;
}
}
void worker( shared(Foo)[] data )
{
data[0].mutate();
//...
}
void main()
{
auto data = new shared(Foo)[10];
spawn( worker, data );
}
--
BR, Ruslan Mullakhmetov
Re: pass array of objects to spawn
On 2011-11-11 01:23:01 +0400, Timon Gehr said:
class Foo
{
}
void worker( shared(Foo[]) data_ )
{
Foo[] data = cast() data_; // this cast is valid because data_ is
never read from another thread after the cast
//...
}
void main()
{
{
auto data = new Foo[10];
spawn( worker, cast(shared)data ); // this cast is valid
because data is an unique reference (therefore there are no unshared
aliases)
} // the sole reference to data in the main thread dies - it will
never be read from this thread again
}
Thank you too. Unfortunately i got compilation error
thread.d(16): Error: cannot implicitly convert expression (data_) of
type shared(Foo)[] to Foo[]
--
BR, Ruslan Mullakhmetov
Re: pass array of objects to spawn
On 2011-11-11 02:48:52 +0400, Timon Gehr said:
On 11/10/2011 11:00 PM, Ruslan Mullakhmetov wrote:
On 2011-11-11 01:23:01 +0400, Timon Gehr said:
class Foo
{
}
void worker( shared(Foo[]) data_ )
{
Foo[] data = cast() data_; // this cast is valid because data_ is
never read from another thread after the cast
//...
}
void main()
{
{
auto data = new Foo[10];
spawn( worker, cast(shared)data ); // this cast is valid because data
is an unique reference (therefore there are no unshared aliases)
} // the sole reference to data in the main thread dies - it will
never be read from this thread again
}
Thank you too. Unfortunately i got compilation error
thread.d(16): Error: cannot implicitly convert expression (data_) of
type shared(Foo)[] to Foo[]
Interesting, apparently cast() does not remove shared. Sorry about
that, use this (this time I tested it).
import std.concurrency;
class Foo
{
}
void worker( shared(Foo[]) data_ )
{
auto data = cast(Foo[]) data_;
//...
}
void main()
{
{
auto data = new Foo[10];
spawn( worker, cast(shared)data );
}
}
thank you very much. For now it's working. But it would be quite
interesting to add explicit ownership semantics to the language as Ali
sugested once by keyword unique.
By the way, I realized that it would be better to create data in thread
rather pass it to thread and pass some seeds for creating data. Of
course, this is one case, sometimes transfer of ownership is not
avoidable as it seems to me.
--
BR, Ruslan Mullakhmetov
bug in std.array.insertInPlace ?
Hi, I found some strange behavior of std.array.insertInPlace leading to segfault. The example below works for int, it is failed for reference type with array becoming of length bigger 1024 on windows (x64) and 512 on linux (x64). code: http://cloud.theambient.org/0O360r1d2t1g09171F1m Is it my bug or compiler? -- BR, Ruslan Mullakhmetov
Re: bug in std.array.insertInPlace ?
On 2011-11-09 22:14:25 +0400, Jonathan M Davis said: On Tuesday, November 08, 2011 23:55 Ruslan Mullakhmetov wrote: Hi, I found some strange behavior of std.array.insertInPlace leading to segfault. The example below works for int, it is failed for reference type with array becoming of length bigger 1024 on windows (x64) and 512 on linux (x64). code: http://cloud.theambient.org/0O360r1d2t1g09171F1m Is it my bug or compiler? It might by related to http://d.puremagic.com/issues/show_bug.cgi?id=6874 - Jonathan M Davis thakns. so i won't submit an issue. -- BR, Ruslan Mullakhmetov
