core.sync.rwmutex example
The example code from core.sync.rwmutex seems bugged. After copying it
I added an import for core.sync.rwmutex, and moved the executions of
runTest into...well:
void main()
{runTest(ReadWriteMutex.Policy.PREFER_READERS);
runTest(ReadWriteMutex.Policy.PREFER_WRITERS);
}
Then I tried to compile it. I got the following error messages:
test3.d(36): Error: class core.sync.rwmutex.ReadWriteMutex member
m_commonMutex is not accessible
test3.d(38): Error: class core.sync.rwmutex.ReadWriteMutex member
m_numQueuedReaders is not accessible
test3.d(39): Error: class core.sync.rwmutex.ReadWriteMutex member
m_numQueuedWriters is not accessible
Checking out the documentation, I don't see that they SHOULD be
accessible, so I think the compiler's correct, and the example is wrong.
P.S.: Does anyone have a good working example of rwmutex? I'm trying
to build a hash table that is write accessible from one thread, and
readable from anywhere, and this looked like the best way to do
it...except that when I start to figure out how to use it I get errors.
Also does anyone have any examples of two directional communication
between two particular threads (a bit more than just yes/no) in the
presence of multiple other threads, so that when a thread asks another
for information, only that other thread is allowed to reply? Perhaps
that's a better way to implement the shared-read hash table. (I'd like
to use std.concurrency, but I can't figure out how one is supposed to
manage specific inter-thread communications.)
--
Charles Hixson
Re: core.sync.rwmutex example
On 05/09/2014 02:51 PM, Joshua Niehus via Digitalmars-d-learn wrote:
Hi Charles,
would the following work (just a shot in the dark) ?
//---
module test;
import std.stdio;
import std.concurrency;
void spawnedFuncFoo(Tid tid, Tid tidBar) {
receive(
(int i) {
writeln("Foo Received the number ", i);
send(tidBar, i, thisTid);
auto barSuccessful = receiveOnly!(string);
writeln("Bar got my (Foo) message");
}
);
send(tid, true);
}
void spawnedFuncBar(Tid tid) {
receive(
(int i, Tid tidFoo) {
writeln("Foo passed me (Bar) the number ", i);
send(tidFoo, "done");
}
);
receive(
(string sig) {
writeln("Main says I'm (Bar) done.");
send(tid, 42);
}
);
}
void main() {
auto tidBar = spawn(&spawnedFuncBar, thisTid);
auto tidFoo = spawn(&spawnedFuncFoo, thisTid, tidBar);
send(tidFoo, 42);
auto fooWasSuccessful = receiveOnly!(bool);
assert(fooWasSuccessful);
send(tidBar, "your done");
auto barWasSuccessful = receiveOnly!(int);
assert(barWasSuccessful == 42);
writeln("Successfully had two separate threads communicate
with each other");
}
//---
Thank you very much for the response.
How do you ensure that the received message (in "auto barWasSuccessful =
receiveOnly!(int); ") came from the thread sent to by tidFoo? Also, why
two receives in
spawnedFuncBar? I thought that receive handled multiple different
arguments with a single receive? Are you asserting that the arguments
happen in sequence, as it appears? That doesn't look as if it would
scale will to multiple threads. And with multiple threads I would
expect the mailbox to start getting very full with inappropriate
messages that never got handled, because they came at the wrong time
(and arguably from the wrong sender, as it seems that there is an
assumption that there is no intervening thread sending any messages).
That's why I was looking for a way to ensure that the message being
received came from the appropriate sender. You can't depend on messages
to always come from the source you expect. This probably means that you
need to have only ONE receive statement, located in a loop. And some
way to check and set the status of incoming messages. Also a yield
method that lets one receive and stack any incoming messages during a
busy cycle, though I think that that's what receive does.
OTOH, I do note that you are using "auto barWasSuccessful =
receiveOnly!(int);" which implies that the value returned is not a void,
even though you never use the result. This fits with the claims of
TDPL, but make me increasingly dubious about the documentation. Or do I
just not understand what a function of type void is supposed to return?
The hashtable itself will be a uint64_t[string] which is confined to one
thread, so the access methods need to be rather straightforwards. I'm
thinking of:
receive
( (string s, uint64_t id) { void set(s, id); }
(string s) { void get(s); }
(Variant v) { void reportError (v); }
);
get should send the id back to the calling thread. set updates the AA
and sends true or false (success) back to the calling thread. But
doesn't this design lock the entire hashtable while the update is in
progress? Is there a better way? set will be, approx:
void set (string s, uint64_t id)
{tbl[s]=id;
send (tid, true);
}
etc. But I'm worried about the receiving end. It needs, somehow, to
ensure that the message it receives is the appropriate message, and that
other messages don't get dropped while it's waiting for the answer...or,
perhaps worse, substituted for the expected answer. If I can depend on
msg[0] of "auto msg = receiveOnly!(Tid, bool)" that will allow me to
check that the message was received from the proper source Also, if a
message is received that if of an inappropriate form, an exception is
thrown. Recovery looks problematic. But this is a case that should be
expected when different threads are sending messages. And if the
message is of the correct form, but from the wrong thread, how can one
re-queue it for later processing?
So far the best idea I've got is to have a dequeue of messages that are
stacked with "receiveOnly!Variant". But this would need to be accesses
in various places throughout the thread, and the handling looks messy.
OTOH, it does let me save messages that come in at the wrong time where
I can deal with them later. So it's not all bad. But it gives up on the
processing that receive makes available. And note that this processing
doesn't need to be done in the hashtable function, that's pretty clear,
but wherever it's used. Which is terrible. As it is, I might be better
putting the hashtable in s synchronized class. The entire hashtable is
getting locked o
Re: core.sync.rwmutex example
On 05/10/2014 06:28 PM, TheFlyingFiddle via Digitalmars-d-learn wrote:
On Friday, 9 May 2014 at 23:12:44 UTC, Charles Hixson via
Digitalmars-d-learn wrote:
But I'm worried about the receiving end. It needs, somehow, to ensure
that the message it receives is the appropriate message, and that
other messages don't get dropped while it's waiting for the
answer...or, perhaps worse, substituted for the expected answer. If
I can depend on msg[0] of "auto msg = receiveOnly!(Tid, bool)" that
will allow me to check that the message was received from the proper
source
If you are worried that other messages having the same signature will
be sent from other sources than the expected source you could make use
of message tagging. Simply wrap the boolean result in a struct with a
descriptive name.
struct SharedHashMapSetCB { bool flag; }
void set (string s, uint64_t id)
{
tbl[s] = id;
send (SharedHashMapSetCB(true));
}
//On the receiving end
auto msg = receiveOnly!SharedHashMapSetCB();
But doesn't this design lock the entire hash-table while the update
is in progress? Is there a better way?
I think a shared memory hash-map is better for your use case. Working
with message passing is preferable done asynchronously. Blocking calls
(send followed by receive) is likely to be slower then simply waiting
on a semaphore.
I think you're probably right. It should be a fairly fast lookup
anyway, and since there should be only one copy of the table, I don't
think one would get much of a speedup no matter what one did. OTOH, it
can't be directly shared. What can be shared is a handle to a singleton
synchronized class. Its requirement that it can be updated, IIUC, means
that it can't be directly shared.
OTOH... I think I may need to build a mailbox class, the design I'm not
yet sure about, that acts as an interface between the major number of
threads and each other. Access to it needs to be non-blocking
(essentially) so it had probably better ONLY act as a message
switchboard. I want it to pass messages with "time" and "origin"
(thread id) prefixed to them. OTOH, I probably only need to support a
small number of message formats, perhaps two, perhaps three. And it
would be more efficient if whenever a thread polled it, it responded
with all waiting mail for that destination. So it's only sending in
response to a received message directed at it, but it holds all messages
directed at some other thread. This allows the other threads to loop
through their processing, only occasionally pausing to check whether
they have any mail or not. I'm hoping a couple of ms will be a long
enough timeout. The mailbox could spend most of it's time sitting
blocked at receive.
This isn't the design I was thinking about earlier, but it seems better,
and it's the design I originally ended up with the last time I thought
about this problem seriously, though admittedly I never got around to
implementing it. The question in my mind is "is it better to build this
on top of std.concurrency, or to go back to the thread class. The code
of std.concurrency shows pretty clearly how to build a mailbox, but the
one that it builds doesn't store the things that I want, and is a lot
more flexible than I have any use for. Still, it's being maintained as
a part of the language...but it's a private class, so I can't use it
directly. But threading is difficult and error prone, so it might be
better to write a layer on top of std.concurrency, even if it means I
need to re-implement a lot of what has already been done.
--
Charles Hixson
Re: Why std.algorithm.sort can't be applied to char[]?
On 05/12/2014 09:29 AM, Jonathan M Davis via Digitalmars-d-learn wrote:
On Mon, 12 May 2014 14:49:52 +
hane via Digitalmars-d-learn wrote:
and is there any way to sort char array with algorithm.sort?
---
import std.algorithm;
import std.range;
void main()
{
int[] arr = [5, 3, 7];
sort(arr); // OK
char[] arr2 = ['z', 'g', 'c'];
sort(arr2); // error
sort!q{ a[0] > b[0] }(zip(arr, arr2)); // error
}
---
I don't know what's difference between int[] and char[] in D, but
it's very unnatural.
All strings in D are treated as ranges of dchar, not their element type. This
has to with the fact that a char or wchar are only part of a character. If you
want to sort arrays of characters, you need to use dchar[].
http://stackoverflow.com/questions/12288465
http://stackoverflow.com/questions/16590650
- Jonathan M Davis
Given that he was working with pure ASCII, he should be able to cast the
array to byte[] and sort it, but I haven't tried. Also char[] isn't
string. Strings are immutable, and thus cannot be sorted in place. To
me this looks like an error in sort that he should be able to work
around with a cast.
--
Charles Hixson
receiveTimeout minimum reasonable timeout value?
Duration can be specified in nanoseconds, but does it make any sense to have a value of 1 nanosecond? 0? My desire is to check whether a message is in the queue, and then either move it local to the thread, or continue, depending. Secondarily, is there much overhead in calling receiveTimeout frequently? It would minimize the storage in the Tid mailbox, but at the cost of requiring more local storage. As I can only receive one message (if there is one) at a time anyway, it's not clear that I would save much by calling it only once/iteration. The current design is that I have an array of classes to tell to process themselves and I have a local AA of messages, with id#s that say which class instance they are intended for. I could either loop through the array, and then loop receiving messages, or I could receive messages in between processing each instance. The messages received may be intended for any instance in the thread, so they get stacked in an AA of the form: Msg[uint64_t][] msg; Msg is immutable and contains both from and to identifiers, among other things not all of which have been decided. (Well, it's a struct of which all fields are immutable, so I presume the struct counts as immutable, though I haven't yet tested it. I may need to redefine it, but it's intended as immutable.) And the only thing passed to receive is a Msg (or a couple of control values in an enum). As an aside, is it better to have more threads than processors? TDPL seems equivocal about this.
Re: receiveTimeout minimum reasonable timeout value?
On Thursday, May 15, 2014 08:21:41 PM JR via Digitalmars-d-learn wrote:
> On Thursday, 15 May 2014 at 18:15:46 UTC, Charles Hixson via
>
> Digitalmars-d-learn wrote:
> > Duration can be specified in nanoseconds, but does it make any
> > sense
> > to have a value of 1 nanosecond? 0?
> >
> > My desire is to check whether a message is in the queue, and
> > then
> > either move it local to the thread, or continue, depending.
>
> I use 0.seconds for such oneshot receive attempts. Snippet:
>
> while (!halt) {
> if (++readCounter > messageCheckFrequency) {
> receiveTimeout(0.seconds,
> &_verbose.fun,
> &_quiet.fun,
> &_unknown.fun
> );
>
> readCounter = 0;
> }
>
> slice = conn.stream.readLine(buf);
> // ...
> }
Thanks. That's what I was hoping to hear.
std.concurrency thread communication problem
I'm building a program which I intend to have many threads that can each send messages to (and receive messages from) each other. The obvious way to do this would be to have a shared array of Tids, but this seems to not work. I'm continually fighting the system to get it to compile, and this makes me think it should probably be done some other way...but what? One possibility is to have each thread maintain a separate array that contains all the threads, which would mean that they would need to be initialized after they were created. This would avoid the problems of shared Tids, but each Tid contains a private mailbox, so this would be being duplicated, and that bothers me...it seems like a poor idea. (Maybe I'm wrong about that...but I don't know.) I do know that I want a n by n communication matrix (leaving out the main thread), with each thread sending messages to all to others. (Well, except for a few that I haven't really defined yet, but which handle separated functions.) My plan was to have each thread run an execution loop which frequently checked for messages received in between performing its own functions. They are not intended to synchronize with each other. They are not intended to be temporary, i.e., each of these threads would be started shortly after program initialization, and continue running until program termination. But how should I get them to know each other's address? I don't want the main thread to need to act as a switchboard between all the others, though I guess that would "sort of" work. (Actually, if I need to do that, that job would be pulled off into yet another thread...and I end up with more threads than processors. Still, that's a design that is possible, IIUC.) Any comments or suggestions?
Re: std.concurrency thread communication problem
On Saturday, May 17, 2014 12:59:22 PM Ali Çehreli via Digitalmars-d-learn
wrote:
> On 05/17/2014 12:33 PM, John Colvin wrote:
> > On Saturday, 17 May 2014 at 18:43:25 UTC, Charles Hixson via
> >
> > Digitalmars-d-learn wrote:
> >> I'm building a program which I intend to have many threads that can
> >> each send
> >> messages to (and receive messages from) each other. The obvious way
> >> to do
> >> this would be to have a shared array of Tids, but this seems to not
> >> work. I'm
> >> continually fighting the system to get it to compile, and this makes
> >> me think
> >> it should probably be done some other way...but what?
> >>
> >> One possibility is to have each thread maintain a separate array that
> >> contains
> >> all the threads, which would mean that they would need to be
> >> initialized after
> >> they were created. This would avoid the problems of shared Tids, but
> >> each Tid
> >> contains a private mailbox, so this would be being duplicated, and that
> >> bothers me...it seems like a poor idea. (Maybe I'm wrong about
> >> that...but I
> >> don't know.)
> >
> > If my understanding is correct, each Tid contains a reference to the
> > corresponding thread's MessageBox (implemented by way of MessageBox
> > being a class), not an independent instance. You should be fine to just
> > have an array of the relevant Tids in each thread.
> >
> > Alternatively, a single __gshared array of threads should work, given
> > you are sufficiently careful with it. Remember, if no-one is doing any
> > writing then you don't need to do any synchronisation of reads.
>
> The following is what I've come up with. I had to use a number of
> shared-related casts.
>
> import std.stdio;
> import std.concurrency;
> import std.datetime;
> import std.random;
> import core.thread;
>
> enum threadCount = 5;
> enum messagePerThread = 3;
>
> // Represents messages sent to threads to start their tasks
> struct Start
> {}
>
> // Receives the number (id) of this thread and the workers to send
> messages to
> void workerFunc(size_t id, shared(Tid)[] workers)
> {
> receiveOnly!Start();
>
> // A local function to reduce code duplication
> bool checkMessageForMe(Duration timeout)
> {
> return receiveTimeout(
> timeout,
> (size_t from) {
> writefln("%s received from %s", id, from);
> });
> }
>
> // My main task is to send messages to others:
> size_t totalSent = 0;
> while (totalSent < messagePerThread) {
> auto to = uniform(0, workers.length);
>
> // Only send to others; not to self
> if (to != id) {
> auto chosen = cast(Tid)workers[to];
> writefln("%s sending to %s", id, to);
> chosen.send(id);
> ++totalSent;
> }
>
> checkMessageForMe(0.seconds);
> }
>
> // Process trailing messages sent to me
> bool received = false;
> do {
> received = checkMessageForMe(10.msecs);
> } while (received);
> }
>
> void main()
> {
> auto workers = new shared(Tid)[threadCount];
>
> foreach (id; 0 .. threadCount) {
> auto worker = spawn(&workerFunc, id, workers);
> workers[id] = cast(shared(Tid))worker;
> }
>
> foreach (sharedWorker; workers) {
> auto worker = cast(Tid)sharedWorker;
> worker.send(Start());
> }
>
> thread_joinAll();
> }
>
> Sample output:
>
> 0 sending to 2
> 4 sending to 3
> 4 sending to 2
> 1 sending to 4
> 3 received from 4
> 3 sending to 2
> 0 sending to 1
> 4 received from 1
> 1 received from 0
> 1 sending to 0
> 0 received from 1
> 0 sending to 1
> 1 received from 0
> 1 sending to 0
> 0 received from 1
> 3 sending to 2
> 4 sending to 2
> 2 sending to 0
> 2 received from 0
> 2 received from 4
> 3 sending to 1
> 2 sending to 3
> 0 received from 2
> 1 received from 3
> 2 received from 3
> 2 sending to 0
> 3 received from 2
> 0 received from 2
> 2 received from 3
> 2 received from 4
>
> Ali
Thank you immensely. That is precisely the kind of information I was hoping
for.
stdout.flush
Using: dmd --version DMD64 D Compiler v2.071.0 on debian Linux, and importing: importstd.stdio; the line: flush(); causes: nt.d(29): Error: undefined identifier 'flush', did you mean function 'fflush'? This appears solved by doing stdout.flush; (compiles, but I'm still writing the code) but as write, writef, etc. don't require explicitly mentioning stdout, I think that the documentation for flush should mention that the file must be specified. Currently it seems to imply that all files will be flushed.
How to get current time as long or ulong?
I've been reading std.datetime documentation backwards and forwards, but if the information is there, I've been missing it. How do I get the current time as a long? Clock.currTime() returns a SysTime, and while currently I can convert that to a long, this is because I looked into the code. What's the supported way? All the documentation seems to be based around auto, which is great if you don't need to store it in memory with a defined number of bits allocated...but lousy if you do. (E.g., I don't want to store a time zone, just the UTC time. What I'm looking for is the opposite of the "FromUnixTime" function.
Re: How to get current time as long or ulong?
On 07/05/2016 11:43 AM, Jonathan M Davis via Digitalmars-d-learn wrote:
On Tuesday, July 05, 2016 11:16:31 Charles Hixson via Digitalmars-d-learn
wrote:
What I'm looking for is the opposite of the "FromUnixTime" function.
SysTime has toUnixTime, which is right above fromUnixTime in the
documentation.
But if what you want is a time_t, and you don't want to deal with SysTime,
there's no point in using std.datetime. Just use core.time to call C's time
function.
- Jonathan M Davis
That's what I want, but I'm worried by the documentation saying:
" This has no relation to the wall clock time, as the wall clock time
can be adjusted (e.g. by NTP), whereas the monotonic clock always moves
forward."
What I want is the system clock time, which *is* adjusted by NTP. I
know it's not guaranteed monotonic, but different computers should have
the same time (or be able to be synchronized to the same NTP time). And
it's "approximately monotonic". time_t is fine, as 32 bit hardware is
becoming scarce, and the application I'm working on will probably never
encounter any. (More than 32 bits of precision is required as I don't
want it to roll over every 42.5 days.)
I understand why some purposes would really want a monotonic time, but
that's not a strong constraint for me. But I do want it to be a long or
ulong. (I don't really care where 0 is. What I'm using now is:
aliaslongClockT;
ClockTnow(){returnMonoTime.currTime();}
And it's working fine during testing, but the documentation about the
setting of MonoTime bothers me.
Re: How to get current time as long or ulong?
I guess I was expressing myself poorly, probably due to muddled thinking
about the representation of time.
Based on various hints from you and others my current guess is that I
should use:
longnow() { returnClock.currTime().stdTime;}
IIUC this should return the current system clock time offset to start at
the start of 1 C.E. (with a standard systematic error that common across
all machines allowing for drift since the last time their clocks were
updated by NTP). And that this is the supported way to get this time.
(Which is probably about the same as MonoTime.currTime(), but *is*
updated when NTP updates the system clock.) And which is standardly a
64 bit number.
On 07/05/2016 11:53 AM, Jonathan M Davis via Digitalmars-d-learn wrote:
On Tuesday, July 05, 2016 11:43:32 Jonathan M Davis via Digitalmars-d-learn
wrote:
But if what you want is a time_t, and you don't want to deal with SysTime,
there's no point in using std.datetime. Just use core.time to call C's time
function.
Actually, I should qualify this. I keep forgetting that time_t is not
guaranteed to be unix time on non-POSIX systems. If what you want is time_t,
then use C's time function via core.time, but if what you want is actual
unix time, then just do Clock.currTime.toUnixTime, since that's the simplest
way to do it. If you're using Microsoft's runtime, then time_t is indeed
unix time, but it's not for the Digital Mars runtime (and the standard -
unfortunately - doesn't require it to be). Digital Mars uses someting that's
_close_ to time_t but isn't actually time_t. :(
- Jonathan M Davis
Re: How to get current time as long or ulong?
Thank you for confirming the change. It hasn't made any difference
during the tests so far, but it sounds like it soon would have.
I don't really want a long...but an int rolls over too quickly, and
there's no 48 bit time. The time is basically for use over smaller
intervals, but occasionally I may need to compare across multiple
months, or even years. So the reason is because I want one time unit to
have multiple uses. It's going to end up being stored on disk, but
order of access is going to frequently be important, and occasionally
the relative age of quite different entities is going to be important.
For the second use I could use a much lower precision timer, but that
would mean using two separate times for each item.
On 07/05/2016 05:10 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
On Tuesday, July 05, 2016 16:18:19 Charles Hixson via Digitalmars-d-learn
wrote:
I guess I was expressing myself poorly, probably due to muddled thinking
about the representation of time.
Based on various hints from you and others my current guess is that I
should use:
longnow() { returnClock.currTime().stdTime;}
IIUC this should return the current system clock time offset to start at
the start of 1 C.E. (with a standard systematic error that common across
all machines allowing for drift since the last time their clocks were
updated by NTP). And that this is the supported way to get this time.
(Which is probably about the same as MonoTime.currTime(), but *is*
updated when NTP updates the system clock.) And which is standardly a
64 bit number.
MonoTime and stdTime have _nothing_ to do with each other. stdTime is the
number of hecto-nanoseconds since midnight, January 1st, 1 A.D. MonoTime
contains a timestamp in system ticks, and how many of those there are per
second as well as how long ago 0 was are completely system-dependent.
The monotonic time is used for timing things not for anything related to the
wall clock. If what you care about is the wall clock, then use SysTime. If
what you care about is timing stuff, then use MonoTime.
I don't know why you'd want a long for the system time. There's nothing
standard about stdTime (which is why it's a horrible name, but
unfortunately, there really isn't a good name for it like there is with unix
time). SysTime exposes stdTime for the rare code that actually needs it, but
in general, code should either be using SysTime or converting SysTime to
the ISO or ISO extended formats, because those are standard. But you
certainly can store the stdTime somewhere if you want to.
- Jonathan M Davis
Re: How to get current time as long or ulong?
On 07/05/2016 05:23 PM, Jonathan M Davis via Digitalmars-d-learn wrote:
On Tuesday, July 05, 2016 12:51:54 Charles Hixson via Digitalmars-d-learn
wrote:
On 07/05/2016 11:43 AM, Jonathan M Davis via Digitalmars-d-learn wrote:
On Tuesday, July 05, 2016 11:16:31 Charles Hixson via Digitalmars-d-learn
wrote:
What I'm looking for is the opposite of the "FromUnixTime" function.
SysTime has toUnixTime, which is right above fromUnixTime in the
documentation.
But if what you want is a time_t, and you don't want to deal with SysTime,
there's no point in using std.datetime. Just use core.time to call C's
time
function.
- Jonathan M Davis
That's what I want, but I'm worried by the documentation saying:
" This has no relation to the wall clock time, as the wall clock time
can be adjusted (e.g. by NTP), whereas the monotonic clock always moves
forward."
What I want is the system clock time, which *is* adjusted by NTP. I
know it's not guaranteed monotonic, but different computers should have
the same time (or be able to be synchronized to the same NTP time). And
it's "approximately monotonic". time_t is fine, as 32 bit hardware is
becoming scarce, and the application I'm working on will probably never
encounter any. (More than 32 bits of precision is required as I don't
want it to roll over every 42.5 days.)
I understand why some purposes would really want a monotonic time, but
that's not a strong constraint for me. But I do want it to be a long or
ulong. (I don't really care where 0 is. What I'm using now is:
aliaslongClockT;
ClockTnow(){returnMonoTime.currTime();}
And it's working fine during testing, but the documentation about the
setting of MonoTime bothers me.
The Monotonic clock has nothing to do with the wall clock. It's giving the
number of system clock ticks which have occurred since some arbitrary time
that the system clock decided was 0, and the number of ticks per second is
completely system dependent. You wouldn't even want to share that number
between programs, let alone between runs of the same programe or between
computers. It's used for timing, not for knowing what time it is. e.g.
auto before = MonoTime.currTime;
// ... do a bunch of stuff
auto after = Monotime.currTime;
// The amount of time that that stuff took
auto diff = after - before;
It's the difference that becomes meaningful. The actual values of before and
after would be meaningless outside of this exact run of the program. What
makes the monotonic clock work for timing unlike the wall clock is that it
always moves forward by a fixed number of ticks per second. It's all about
timing and not at all about determing what time of day it is.
The wall clock time, on the other hand, is all about the time of day. It's
this particular hour, minute, second, etc. on this particular day of this
particular month of this particular year. And because the system's wall
clock time can be changed (be it because of NTP or someone manually changing
the time - or even because of a change in time zone if you're dealing with
the local time), it is not guaranteed to always move forward at a fixed
rate. It could change radically, which is why it does not work for timing.
It's all about knowing what time of day that something happened.
There really should never be a question of whether the monotonic time or the
wall clock time is appropriate. If you're ever asking that question, then
you almost certainly don't understand what they are.
If what you want is to time stuff (e.g. this happened x minutes into the run
of this program or this operation took y seconds), then you want the
monotonic time. If what you want is to know when something happens - i.e.
the time of day and/or date - then what you want is the wall clock time.
What you're using it for should make it obvious which you need.
But I would have to know more about what you're trying to do to have any
clue what you should be using. And I don't know why you would be so
interested in having the time as an integral value. That's usually a bad
idea. It's almost always better to use a time type that actually involves
units and context rather than a naked integral value.
- Jonathan M Davis
The same time needs to be used for two different purposes (or I have to
keep two separate times). One time is used during a particular run of
the program to compare when two different things happened. This needs
fine discrimination...millisecond level. The other needs to be used to
order events happening over a period of time. This needs to be at the
hour level, and the minute level is better. But it needs to work over
years. That's why I originally said UnixTime...and UnixTime would work
fine, but problems might happen if it were in use in 2029(?*). Since D
supports a 64 bit ti
Re: How to get current time as long or ulong?
On 07/06/2016 10:32 AM, H. S. Teoh via Digitalmars-d-learn wrote: On Wed, Jul 06, 2016 at 10:19:19AM -0700, Charles Hixson via Digitalmars-d-learn wrote: [...] The same time needs to be used for two different purposes (or I have to keep two separate times). One time is used during a particular run of the program to compare when two different things happened. This needs fine discrimination...millisecond level. The other needs to be used to order events happening over a period of time. This needs to be at the hour level, and the minute level is better. But it needs to work over years. That's why I originally said UnixTime...and UnixTime would work fine, but problems might happen if it were in use in 2029(?*). Since D supports a 64 bit time, not using it would seem to be unreasonable. I looked into scaling it, but either I use more than 32 bits, or I need to keep two times. So the long version of SysTime seems to be what I want. That will even let me compare things against books published in the Classical Greek period, and doesn't threaten to roll over. It's more precise than I need, but there's no one time of lesser precision that will do everything I want. The problem with this is that the system clock may change. Would your program continue to function correctly if, during execution, the user changes the system clock? Or if the OS in the background syncs with a remote NTP server and loses/gains a few seconds/minutes during which events are happening that record timestamps? I.e., some events might have their timestamps out-of-order with the actual sequence of events if you use system time. T It would err in a trivial way, which might require reloading something from disk. So an occasional mistake of that nature isn't a problem.
How to open file with exclusive lock?
I want to open a file with an exclusive lock. It would be important that no other thread be able to access the file in write mode, and desirable that no other thread be able to access the file in read mode. (Ditto for other processes.) stdio.file.lock (or is it stdio.file.File.lock?) seems to demand a range of chunks to lock, but the file is not fixed in length. Is it appropriate to just specify the maximum possible number of bytes (i.e. ulong.max)?
Re: How to open file with exclusive lock?
On 07/12/2016 12:05 PM, H. S. Teoh via Digitalmars-d-learn wrote: On Tue, Jul 12, 2016 at 11:54:18AM -0700, Charles Hixson via Digitalmars-d-learn wrote: I want to open a file with an exclusive lock. It would be important that no other thread be able to access the file in write mode, and desirable that no other thread be able to access the file in read mode. (Ditto for other processes.) stdio.file.lock (or is it stdio.file.File.lock?) seems to demand a range of chunks to lock, but the file is not fixed in length. Is it appropriate to just specify the maximum possible number of bytes (i.e. ulong.max)? Whether this is even possible depends on your OS. I don't know of any cross-platform way of file-locking that is guaranteed to work everywhere. In Posix there's fcntl(F_RDLCK / F_WRLCK), which I believe is the underlying OS call for File.lock. However, this is only an *advisory* lock, i.e., any other processes accessing the file must also call flock(), otherwise none of its protections apply. It only works between cooperating processes. Linux does have a mandatory locking feature, but it requires the kernel to be specially configured for it, and the filesystem must also support it (and must be mounted with the correct options). Unfortunately it's Linux-specific and probably won't work for any other OS. Windows may have some file-locking mechanism that does what you want, but I'm not familiar with the intricacies of how it works. T OK. It's not possible without OS support. Agreed. And I don't want to get into C calls, but rather to use the mechanisms that D provides. And this *probably* won't cause any problems. But how should I tell D that that's the way I want to access to work? It's not clear to me that locking beyond the current EOF is valid, or whether it will decide that it needs to reserve the space before I use it (in which case locking from 0->ulong.max bytes is a bad idea, and would totally fill my hard disk).
Re: How to open file with exclusive lock?
On 07/12/2016 03:54 PM, H. S. Teoh via Digitalmars-d-learn wrote: On Tue, Jul 12, 2016 at 03:40:36PM -0700, Charles Hixson via Digitalmars-d-learn wrote: [...] OK. It's not possible without OS support. Agreed. And I don't want to get into C calls, but rather to use the mechanisms that D provides. And this *probably* won't cause any problems. But how should I tell D that that's the way I want to access to work? It's not clear to me that locking beyond the current EOF is valid, or whether it will decide that it needs to reserve the space before I use it (in which case locking from 0->ulong.max bytes is a bad idea, and would totally fill my hard disk). Here's another approach. Since it's almost a given that we're talking about cooperating processes sharing an advisory lock here (since a global mandatory file lock seems to have poor support, at least on Posix), we don't *have* to lock the actual bytes we're writing to. We can simply reserve a couple of bytes (perhaps even just one byte) at the beginning of the file, and agree among all processes that we will always try to acquire lock on that byte before writing to (the rest of) the file. Essentially it would serve as a file equivalent of a mutex. Then wrap all your file primitives in a little wrapper that acquires this mutex before performing file I/O, have your program only do I/O through this wrapper, and you're all set. (IIRC, this is how the SQLite3 library implements database read/write locks. Each sqlite3 database file has a few bytes, IIRC 6 bytes, at a known file offset, that it uses to control various levels of access to the database. Depending on which level of locking was needed, it would acquire a lock on one or more of these bytes, before accessing the rest of the file. It doesn't actually acquire a lock on the bytes being read / modified.) T That's probably the most viable approach. It does mean there's a lag between file open and file lock (unless I was to mess around with concurrency locks), but it's quite unlikely that anyone else will access the file while I have it open. (Well, possibly baloo or nepomuk or some such, but that's probably harmless.)
randomIO, std.file, core.stdc.stdio
Are there reasons why one would use rawRead and rawWrite rather than fread and fwrite when doiing binary random io? What are the advantages? In particular, if one is reading and writing structs rather than arrays or ranges, are there any advantages?
Re: randomIO, std.file, core.stdc.stdio
On 07/25/2016 05:18 PM, ketmar via Digitalmars-d-learn wrote: On Monday, 25 July 2016 at 18:54:27 UTC, Charles Hixson wrote: Are there reasons why one would use rawRead and rawWrite rather than fread and fwrite when doiing binary random io? What are the advantages? In particular, if one is reading and writing structs rather than arrays or ranges, are there any advantages? yes: keeping API consistent. ;-) for example, my stream i/o modules works with anything that has `rawRead`/`rawWrite` methods, but don't bother to check for any other. besides, `rawRead` is just looks cleaner, even with all `(&a)[0..1])` noise. so, a question of style. OK. If it's just a question of "looking cleaner" and "style", then I will prefer the core.stdc.stdio approach. I find it's appearance extremely much cleaner...except that that's understating things. I'll probably wrap those routines in a struct to ensure things like files being properly closed, and not have explicit pointers persisting over large areas of code. (I said a lot more, but it was just a rant about how ugly I find rawRead/rawWrite syntax, so I deleted it.)
Re: randomIO, std.file, core.stdc.stdio
On 07/25/2016 07:11 PM, ketmar via Digitalmars-d-learn wrote: On Tuesday, 26 July 2016 at 01:19:49 UTC, Charles Hixson wrote: then I will prefer the core.stdc.stdio approach. I find it's appearance extremely much cleaner... only if you are really used to write C code. when you see pointer, or explicit type size argument in D, it is a sign of C disease. I'll probably wrap those routines in a struct to ensure things like files being properly closed, and not have explicit pointers persisting over large areas of code. exactly what std.stdio.File did! ;-) Yes, but I really despise the syntax they came up with. It's probably good if most of your I/O is ranges, but mine hasn't yet ever been. (Combining ranges with random I/O?)
Re: randomIO, std.file, core.stdc.stdio
On 07/25/2016 09:22 PM, ketmar via Digitalmars-d-learn wrote:
On Tuesday, 26 July 2016 at 04:05:22 UTC, Charles Hixson wrote:
Yes, but I really despise the syntax they came up with. It's
probably good if most of your I/O is ranges, but mine hasn't yet ever
been. (Combining ranges with random I/O?)
that's why i wrote iv.stream, and then iv.vfs, with convenient things
like `readNum!T`, for example. you absolutely don't need to
reimplement the whole std.stdio.File if all you need it better API.
thanks to UFCS, you can write your new API as free functions accepting
std.stdio.File as first arg. or even generic stream, like i did in
iv.stream:
enum isReadableStream(T) = is(typeof((inout int=0) {
auto t = T.init;
ubyte[1] b;
auto v = cast(void[])b;
t.rawRead(v);
}));
enum isWriteableStream(T) = is(typeof((inout int=0) {
auto t = T.init;
ubyte[1] b;
t.rawWrite(cast(void[])b);
}));
T readInt(T : ulong, ST) (auto ref ST st) if (isReadableStream!ST) {
T res;
ubyte* b = cast(ubyte*)&res;
foreach (immutable idx; 0..T.sizeof) {
if (st.rawRead(b[idx..idx+1]).length != 1) throw new
Exception("read error");
}
return res;
}
and then:
auto fl = File("myfile");
auto i = fl.readInt!uint;
something like that.
That's sort of what I have in mind, but I want to do what in Fortran
would be (would have been?) called record I/O, except that I want a file
header that specifies a few things like magic number, records allocated,
head of free list, etc. In practice I don't see any need for record
size not known at compile time...except that if there are different
versions of the program, they might include different things, so, e.g.,
the size of the file header might need to be variable.
This is a design problem I'm still trying to wrap my head around.
Efficiency seems to say "you need to know the size at compile time", but
flexibility says "you can't depend on the size at compile time". The
only compromise position seems to compromise safety (by depending on
void * and record size parameters that aren't guaranteed safe). I'll
probably eventually decide in favor of "size fixed at compile time", but
I'm still dithering. But clearly efficiency dictates that the read size
not be a basic type. I'm currently thinking of a struct that's about 1
KB in size. As far as the I/O routines are concerned this will probably
all be uninterpreted bytes, unless I throw in some sequencing for error
recovery...but that's probably making things too complex, and should be
left for a higher level.
Clearly this is a bit of a specialized case, so I wouldn't be
considering implementing all of stdio, only the relevant bits, and those
wrapped with an interpretation based around record number.
The thing is, I'd probably be writing this wrapper anyway, what I was
wondering originally is whether there was any reason to use std.file as
the underlying library rather than going directly to core.stdc.stdio.
Re: randomIO, std.file, core.stdc.stdio
On 07/26/2016 05:31 AM, Steven Schveighoffer via Digitalmars-d-learn wrote:
On 7/25/16 9:19 PM, Charles Hixson via Digitalmars-d-learn wrote:
On 07/25/2016 05:18 PM, ketmar via Digitalmars-d-learn wrote:
On Monday, 25 July 2016 at 18:54:27 UTC, Charles Hixson wrote:
Are there reasons why one would use rawRead and rawWrite rather than
fread and fwrite when doiing binary random io? What are the
advantages?
In particular, if one is reading and writing structs rather than
arrays or ranges, are there any advantages?
yes: keeping API consistent. ;-)
for example, my stream i/o modules works with anything that has
`rawRead`/`rawWrite` methods, but don't bother to check for any other.
besides, `rawRead` is just looks cleaner, even with all `(&a)[0..1])`
noise.
so, a question of style.
OK. If it's just a question of "looking cleaner" and "style", then I
will prefer the core.stdc.stdio approach. I find it's appearance
extremely much cleaner...except that that's understating things. I'll
probably wrap those routines in a struct to ensure things like files
being properly closed, and not have explicit pointers persisting over
large areas of code.
It's more than just that. Having a bounded array is safer than a
pointer/length separated parameters. Literally, rawRead and rawWrite
are inferred @safe, whereas fread and fwrite are not.
But D is so nice with UFCS, you don't have to live with APIs you don't
like. Allow me to suggest adding a helper function to your code:
rawReadItem(T)(File f, ref T item) @trusted
{
f.rawRead(&item[0 .. 1]);
}
-Steve
That *does* make the syntax a lot nicer, and I understand the safety
advantage of not using pointer/length separated parameters. But I'm
going to be wrapping the I/O anyway, and the external interface is going
to be more like:
struct RF (T, long magic)
{
void read (size_t recNo, ref T val){...}
size_t read (ref T val){...}
...
}
where a sequential read returns the record number, or you specify the
record number and get an indexedIO read. So the length with be
T.sizeof, and will be specified at the time the file is opened. To me
this seems to eliminate the advantage of stdfile, and stdfile seems to
add a level of indirection.
Ranges aren't free, are they? If so then I should probably use stdfile,
because that is probably less likely to change than core.stdc.stdio.
When I see "f.rawRead(&item[0 .. 1])" it looks to me as if unneeded code
is being generated explictly to be thrown away. (I don't like using
pointer/length either, but it's actually easier to understand than this
kind of thing, and this LOOKS like it's generating extra code.)
That said, perhaps I should use stdio anyway. When doing I/O it's the
disk speed that's the really slow part, and that so dominates things
that worrying about trivialities is foolish. And since it's going to be
wrapped anyway, the ugly will be confined to a very small routine.
Re: randomIO, std.file, core.stdc.stdio
On 07/26/2016 10:18 AM, Steven Schveighoffer via Digitalmars-d-learn wrote: On 7/26/16 12:58 PM, Charles Hixson via Digitalmars-d-learn wrote: Ranges aren't free, are they? If so then I should probably use stdfile, because that is probably less likely to change than core.stdc.stdio. Do you mean slices? When I see "f.rawRead(&item[0 .. 1])" it looks to me as if unneeded code is being generated explictly to be thrown away. (I don't like using pointer/length either, but it's actually easier to understand than this kind of thing, and this LOOKS like it's generating extra code.) This is probably a misunderstanding on your part. &item is accessing the item as a pointer. Since the compiler already has it as a reference, this is a noop -- just an expression to change the type. [0 .. 1] is constructing a slice out of a pointer. It's done all inline by the compiler (there is no special _d_constructSlice function), so that is very very quick. There is no bounds checking, because pointers do not have bounds checks. So there is pretty much zero overhead for this. Just push the pointer and length onto the stack (or registers, not sure of ABI), and call rawRead. That said, perhaps I should use stdio anyway. When doing I/O it's the disk speed that's the really slow part, and that so dominates things that worrying about trivialities is foolish. And since it's going to be wrapped anyway, the ugly will be confined to a very small routine. Having written a very templated io library (https://github.com/schveiguy/iopipe), I can tell you that in my experience, the slowdown comes from 2 things: 1) spending time calling the kernel, and 2) not being able to inline. This of course assumes that proper buffering is done. Buffering should mitigate most of the slowdown from the disk. It is expensive, but you amortize the expense by buffering. C's i/o is pretty much as good as it gets for an opaque non-inlinable system, as long as your requirements are simple enough. The std.stdio code should basically inline into the calls you should be making, and it handles a bunch of stuff that optimizes the calls (such as locking the file handle for one complex operation). -Steve Thanks. Since there isn't any excess overhead I guess I'll use stdio. Buffering, however, isn't going to help at all since I'm doing randomIO. I know that most of the data the system reads from disk is going to end up getting thrown away, since my records will generally be smaller than 8K, but there's no help for that.
Re: randomIO, std.file, core.stdc.stdio
On 07/26/2016 11:31 AM, Steven Schveighoffer via Digitalmars-d-learn wrote: On 7/26/16 1:57 PM, Charles Hixson via Digitalmars-d-learn wrote: Thanks. Since there isn't any excess overhead I guess I'll use stdio. Buffering, however, isn't going to help at all since I'm doing randomIO. I know that most of the data the system reads from disk is going to end up getting thrown away, since my records will generally be smaller than 8K, but there's no help for that. Even for doing random I/O buffering is helpful. It depends on the size of your items. Essentially, to read 10 bytes from a file probably costs the same as reading 100,000 bytes from a file. So may as well buffer that in case you need it. Now, C i/o's buffering may not suit your exact needs. So I don't know how it will perform. You may want to consider mmap which tells the kernel to link pages of memory directly to disk access. Then the kernel is doing all the buffering for you. Phobos has support for it, but it's pretty minimal from what I can see: http://dlang.org/phobos/std_mmfile.html -Steve I've considered mmapfile often, but when I read the documentation I end up realizing that I don't understand it. So I look up memory mapped files in other places, and I still don't understand it. It looks as if the entire file is stored in memory, which is not at all what I want, but I also can't really believe that's what's going on. I know that there was an early form of this in a version of BASIC (the version that RISS was written in, but I don't remember which version that was) and in *that* version array elements were read in as needed. (It wasn't spectacularly efficient.) But memory mapped files don't seem to work that way, because people keep talking about how efficient they are. Do you know a good introductory tutorial? I'm guessing that "window size" might refer to the number of bytes available, but what if you need to append to the file? Etc. A part of the problem is that I don't want this to be a process with an arbitrarily high memory use. Buffering would be fine, if I could use it, but for my purposes sequential access is likely to be rare, and the working layout of the data in RAM doesn't (can't reasonably) match the layout on disk. IIUC (this is a few decades old) the system buffer size is about 8K. I expect to never need to read that large a chunk, but I'm going to try to keep the chunks in multiples of 1024 bytes, and if it's reasonable to exactly 1024 bytes. So I should never need two reads or writes for a chunk. I guess to be sure of this I'd better make sure the file header is also 1024 bytes. (I'm guessing that the seek to position results in the appropriate buffer being read into the system buffer, so if my header were 512 bytes I might occasionally need to do double reads or writes.) I'm guessing that memory mapped files trade off memory use against speed of access, and for my purposes that's probably a bad trade, even though databases are doing that more and more. I'm likely to need all the memory I can lay my hands on, and even then thrashing wouldn't surprise me. So a fixed buffer size seems a huge advantage.
Re: randomIO, std.file, core.stdc.stdio
On 07/26/2016 12:53 PM, Adam D. Ruppe via Digitalmars-d-learn wrote:
On Tuesday, 26 July 2016 at 19:30:35 UTC, Charles Hixson wrote:
It looks as if the entire file is stored in memory, which is not at
all what I want, but I also can't really believe that's what's going on.
It is just mapped to virtual memory without actually being loaded into
physical memory, so when you access the array it returns, the kernel
loads a page of the file into memory, but it doesn't do that until it
actually has to.
Think of it as being like this:
struct MagicFile {
ubyte[] opIndex(size_t idx) {
auto buffer = new ubyte[](some_block_length);
fseek(fp, idx, SEEK_SET);
fread(buffer.ptr, buffer.length, 1);
return buffer;
}
}
And something analogous for writing, but instead of being done with
overloaded operators in D, it is done with the MMU hardware by the
kernel (and the kernel also does smarter buffering than this little
example).
A part of the problem is that I don't want this to be a process with
an arbitrarily high memory use.
The kernel will automatically handle physical memory usage too,
similarly to a page file. If you haven't read a portion of the file
recently, it will discard that page, since it can always read it again
off disk if needed, but if you do have memory to spare, it will keep
the data in memory for faster access later.
So basically the operating system handles a lot of the details which
makes it efficient.
Growing a memory mapped file is a bit tricky though, you need to unmap
and remap. Since it is an OS concept, you can always look for C or C++
examples too, like herE:
http://stackoverflow.com/questions/4460507/appending-to-a-memory-mapped-file/4461462#4461462
O, dear. It was sounding like such an excellent approach until this
last paragraph, but growing the file is going to be one of the common
operations. (Certainly at first.) It sounds as if that means the file
needs to be closed and re-opened for extensions. And I quote from
https://www.gnu.org/software/libc/manual/html_node/Memory_002dmapped-I_002fO.html:
Function: /void */ *mremap* /(void *address, size_t length, size_t
new_length, int flag)/
Preliminary: | MT-Safe | AS-Safe | AC-Safe | See POSIX Safety
Concepts
.
This function can be used to change the size of an existing memory
area. address and length must cover a region entirely mapped in the
same |mmap| statement. A new mapping with the same characteristics
will be returned with the length new_length.
...
This function is only available on a few systems. Except for performing
optional optimizations one should not rely on this function.
END
So I'm probably better off sticking to using a seek based i/o system.
Re: randomIO, std.file, core.stdc.stdio
On 07/27/2016 06:46 AM, Rene Zwanenburg via Digitalmars-d-learn wrote: On Wednesday, 27 July 2016 at 02:20:57 UTC, Charles Hixson wrote: O, dear. It was sounding like such an excellent approach until this last paragraph, but growing the file is going to be one of the common operations. (Certainly at first.) (...) So I'm probably better off sticking to using a seek based i/o system. Not necessarily. The usual approach is to over-allocate your file so you don't need to grow it that often. This is the exact same strategy used by D's dynamic arrays and grow-able array-backed lists in other languages - the difference between list length and capacity. There is no built-in support for this in std.mmfile afaik. But it's not hard to do yourself. Well, that would mean I didn't need to reopen the file so often, but that sure wouldn't mean I wouldn't need to re-open the file. And it would add considerable complexity. Possibly that would be an optimal approach once the data was mainly collected, but I won't want to re-write this bit at that point.
arrays, mmu, addressing choices
I have a rather large array that I intend to build. but much of it will only occasionally be used. Will the unused sections automatically be paged out? If it matters my system is Debian Linux. This array will be indexed by a ulong. Is there any reasonable maximum size? I've considered segmented addressing anyway, in case that's needed to allow paging, and it will definitely be needed when I get around to concurrent processing, with different sections resident in different threads. But if appropriate I could do that from the start. The questions above are really for a normal array, but I'd also be interested in how using an associative array would affect them. I expect to eventually be using more memory than I have RAM in my system, so designing for paging is going to be important. (Before then I'll be adding more RAM and a larger disk drive, but if I complete the full design even a large disk is going to be small.)
how to declare an immutable class?
I want to declare a class all instances of which will be immutable, and
all references to which will be inherently immutable (so that I don't
need to slip a huge number of "immutable" statements in my code).
This is surely possible, because string acts just that way, but I can't
figure out how to do this.
immutable classMsg {this(...) immutable{...} ... }
doesn't work that way, as when I do
Msg m = new Msg (...);
I get:
Error: incompatible types for ((this.m) - (m)): 'immutable(Msg)' and
'cellram.Msg'
and
Error: immutable method cellram.Msg.this is not callable using a mutable
object
Does anyone know the correct approach?
Re: how to declare an immutable class?
A way around this, which may be the same as the approach used by string was:
alias immutable(Msg_)Msg;
classMsg_
{ ...
This so far appears to do what I want. The only problem is that it
introduces an extraneous symbol, which I would prefer to avoid.
OTOH, I did fix a few problems before this solution
On 08/11/2016 10:56 AM, Charles Hixson via Digitalmars-d-learn wrote:
I want to declare a class all instances of which will be immutable,
and all references to which will be inherently immutable (so that I
don't need to slip a huge number of "immutable" statements in my code).
This is surely possible, because string acts just that way, but I
can't figure out how to do this.
immutable classMsg {this(...) immutable{...} ... }
doesn't work that way, as when I do
Msg m = new Msg (...);
I get:
Error: incompatible types for ((this.m) - (m)): 'immutable(Msg)' and
'cellram.Msg'
and
Error: immutable method cellram.Msg.this is not callable using a
mutable object
Does anyone know the correct approach?
Re: how to declare an immutable class?
On 08/11/2016 06:33 PM, Mike Parker via Digitalmars-d-learn wrote:
On Friday, 12 August 2016 at 00:44:31 UTC, Charles Hixson wrote:
A way around this, which may be the same as the approach used by
string was:
alias immutable(Msg_)Msg;
classMsg_
{ ...
This is exactly what Jonathan suggested in the post above. And yes,
it's how string is handled:
alias string = immutable(char)[];
This so far appears to do what I want. The only problem is that it
introduces an extraneous symbol, which I would prefer to avoid.
What problem do you want to avoid? This is a common D idiom. Alias
names are very short-lived. Yes, it's another symbol in the namespace,
but it will never make it to the object file. It won't even make it to
error messages -- you'll see Msg_ there instead.
Yes, I saw his answer after I'd posted my resolution.
It works, it's just not the syntax that I'd prefer. And it leaves me
wondering exactly what
immutable class Msg {...}
was declaring.
Re: how to declare an immutable class?
Thank you both.
On 08/12/2016 01:35 AM, Jonathan M Davis via Digitalmars-d-learn wrote:
On Friday, August 12, 2016 05:25:45 Mike Parker via Digitalmars-d-learn wrote:
immutable class Foo { ... } is the same as declaring every member
of Foo as immutable, just as final class Foo { ... } makes every
method final.
I'm not sure that that's quite the same thing, because there is such a thing
as a final class, because making the class final makes it illegal to derive
another class from it rather than just affecting the class' functions. So,
final does affect the class itself, whereas immutable does not. It's the
same with most of attributes though - @safe, pure, nothrow, etc. They affect
the members but not the class itself. Putting them on the class is the same
as doing
attribute
{
class C
{
...
}
}
or the same as
attribute:
class C
{
...
}
if there's nothing after the class in the file (since the label syntax
affects everything after it in the file, whereas the braces only affect
what's in the braces).
- Jonathan M Davis
Multi-Thread message passing approach
This is an approach to n x n thread message passing. The idea is that
each thread should be able to pass messages to any other thread. The
only alternative I've come up with involves the main thread handling
each message. Is that a better approach? Is there a better way to pass
lists of Tids?
importstd.concurrency;
importstd.stdio;
importcore.thread;
sharedTid[3]tidList;
structStart{intdummy=0;}
structMsg{intorig;intdest;}
structDone{intdummy=0;}
voidworker (int ndx)
{
writeln("worker ", ndx, " spawned");
{ automsg=receiveOnly!(Start)();
writeln("worker ", ndx, " started");
}
Tid[]tids;
foreach(t; tidList){tids~=cast(Tid)t;}
for(int i = 0;i < 3;i++)
{ if(i != ndx)
{ Msg msg=Msg(ndx, i);
send (tids[i], msg);
}
}
writeln("worker ", ndx, " got tidList");
booldone=false;
while(!done)
{ receive
( (Msg msg){writeln ("msg from: ", msg.orig, ", to:
", msg.dest);},
(Done d){done = true;}
);
}
writeln ("worker ", ndx, " is done");
}
voidmain()
{
Startstart;
Donedone;
for(int i = 0;i < 3;i++)
{ auto tid=spawn (&worker, i);
tidList[i]=cast(shared Tid)tid;
}
foreach (i; 0 .. 3){send (cast(Tid)tidList[i], start);}
Thread.sleep (1000.msecs);
foreach (i; 0 .. 3){send (cast(Tid)tidList[i], done);}
Thread.sleep (1000.msecs);
writeln ("main is done");
}
Re: Multi-Thread message passing approach
On 08/14/2016 07:44 AM, Charles Hixson via Digitalmars-d-learn wrote:
This is an approach to n x n thread message passing. The idea is that
each thread should be able to pass messages to any other thread. The
only alternative I've come up with involves the main thread handling
each message. Is that a better approach? Is there a better way to
pass lists of Tids?
An attempt using the main thread as a relay follows. I have fewer
doubts about it working, as Tids aren't accessed by multiple threads,
but the logic is more convoluted, and harder to get right.
import std.concurrency;
import std.stdio;
import core.thread;
struct Start { int dummy = 0; }
struct Msg { int orig; int dest; }
struct Done { int dummy = 0; }
enum threadCnt = 3;
void worker (int ndx)
{
writeln ("worker ", ndx, " spawned");
{ auto msg = receiveOnly!(Start)();
writeln ("worker ", ndx, " started");
}
Tid owner = ownerTid;
for (int i = 0; i < threadCnt; i++)
{ if (i != ndx)
{ Msg msg = Msg(ndx, i);
send (owner, msg);
}
}
writeln ("worker ", ndx, " sent msgs");
bool done = false;
while (!done)
{ receive
( (Msg msg) { writeln ("msg from: ", msg.orig, ", to: ",
msg.dest); },
(Done d) { done = true; writeln ("thread ", ndx, " is
done."); }
);
}
send (owner, Done(ndx));
writeln ("worker ", ndx, " is done");
}
void main()
{ Tid[] tidList;
Start start;
Done done;
for (int i = 0; i < threadCnt; i++)
{ tidList ~=spawn (&worker, i);
}
yield;
foreach (i; 0 .. threadCnt) { send (tidList[i], start); }
int tidLength = cast(int)tidList.length;
Thread.sleep (500.msecs);
foreach (i; 0 .. threadCnt) { send (tidList[i], done); }
while (tidLength > 0)
{ receive
( (Msg msg) { send (tidList[msg.dest], msg); },
(Done d) { writeln ("tid ", d.dummy, " is done..",
--tidLength, " remaining."); }
);
}
writeln ("main is done");
}
Re: Multi-Thread message passing approach
Looking at the std.concurrency code, it appears that Tid is just a
handle to a class, so multiple assignments should all refer to the same
underlying class, and it looks like that underlying class (MessageBox)
uses mutexes to ensure safe handling of multiple access. So this shared
access to tids should be safe. But in that case why can't Tids be
passed as function parameters to spawn, which would have avoided the
requirement for a fixed length shared array? So I don't trust my
reasoning here.
If I modify the code to attempt to pass a Tid[] as a member of struct
Start I get:
/usr/include/dmd/phobos/std/concurrency.d(603): Error: static assert
"Aliases to mutable thread-local data not allowed."
test.d(47):instantiated from here: send!(Start)
and this seems to imply that the casting away of shared might also be an
unsafe access which just doesn't happen to be detected by the library.
On 08/14/2016 07:44 AM, Charles Hixson via Digitalmars-d-learn wrote:
This is an approach to n x n thread message passing. The idea is that
each thread should be able to pass messages to any other thread. The
only alternative I've come up with involves the main thread handling
each message. Is that a better approach? Is there a better way to
pass lists of Tids?
importstd.concurrency;
importstd.stdio;
importcore.thread;
sharedTid[3]tidList;
structStart{intdummy=0;}
structMsg{intorig;intdest;}
structDone{intdummy=0;}
voidworker (int ndx)
{
writeln("worker ", ndx, " spawned");
{ automsg=receiveOnly!(Start)();
writeln("worker ", ndx, " started");
}
Tid[]tids;
foreach(t; tidList){tids~=cast(Tid)t;}
for(int i = 0;i < 3;i++)
{ if(i != ndx)
{ Msg msg=Msg(ndx, i);
send (tids[i], msg);
}
}
writeln("worker ", ndx, " got tidList");
booldone=false;
while(!done)
{ receive
( (Msg msg){writeln ("msg from: ", msg.orig, ", to:
", msg.dest);},
(Done d){done = true;}
);
}
writeln ("worker ", ndx, " is done");
}
voidmain()
{
Startstart;
Donedone;
for(int i = 0;i < 3;i++)
{ auto tid=spawn (&worker, i);
tidList[i]=cast(shared Tid)tid;
}
foreach (i; 0 .. 3){send (cast(Tid)tidList[i], start);}
Thread.sleep (1000.msecs);
foreach (i; 0 .. 3){send (cast(Tid)tidList[i], done);}
Thread.sleep (1000.msecs);
writeln ("main is done");
}
Re: Multi-Thread message passing approach
I misunderstood the problem. The problem was that a dynamically sized
array cannot be sent as a message. So this works:
import std.concurrency;
import std.stdio;
import core.thread;
enum tidMax = 10;
struct Start { int tidCnt = 0; Tid[tidMax] tids; }
struct Msg { int orig; int dest; }
struct Done { int dummy = 0; }
void worker (int ndx)
{
writeln ("worker ", ndx, " spawned");
Start start = receiveOnly!(Start)();
Tid[] tids;
foreach (i; 0 .. start.tidCnt) { tids ~= start.tids[i]; }
writeln ("worker ", ndx, " got tidList");
for (int i = 0; i < 3; i++)
{ if (i != ndx)
{ Msg msg = Msg(ndx, i);
send (tids[i], msg);
}
}
writeln ("worker ", ndx, " sent messages");
bool done = false;
while (!done)
{ receive
( (Msg msg) { writeln ("msg from: ", msg.orig, ", to: ",
msg.dest); },
(Done d) { done = true; }
);
}
writeln ("worker ", ndx, " is done");
}
void main()
{
Start start;
Done done;
for (int i = 0; i < 3; i++)
{ auto tid = spawn (&worker, i);
start.tids[start.tidCnt++] = tid;
}
foreach (i; 0 .. start.tidCnt) { send (start.tids[i], start); }
Thread.sleep (1000.msecs);
foreach (i; 0 .. start.tidCnt) { send (start.tids[i], done); }
Thread.sleep (1000.msecs);
writeln ("main is done");
}
Re: Multi-Thread message passing approach
On 08/16/2016 07:21 AM, Kagamin via Digitalmars-d-learn wrote: On Monday, 15 August 2016 at 01:53:33 UTC, Charles Hixson wrote: If I modify the code to attempt to pass a Tid[] as a member of struct Start I get: /usr/include/dmd/phobos/std/concurrency.d(603): Error: static assert "Aliases to mutable thread-local data not allowed." test.d(47):instantiated from here: send!(Start) and this seems to imply that the casting away of shared might also be an unsafe access which just doesn't happen to be detected by the library. Currently not all types, that are supposed to be thread-safe, support shared, e.g. Semaphore, so it doesn't necessarily mean that sharing them is incorrect. Thanks. But "not all types" is a bit worrisome. Fortunately it's just as easy to pass a fixed length array of Tids through send, and that even yields simpler code. The problems I was running into turned out to be because there was a "local reference", i.e. the dynamic array. I just wasn't understanding the error message. So now I don't need to use the "shared data" that I was worrying about. I am a bit surprised, however, as Tids include a private class reference, and it seems as if that should raise the same error flag, but it doesn't. Of course, the real problem is that just because concurrent software seems to be working right when you test it doesn't say anything about how it will perform under load...so I want to avoid anything even questionable.
strange -fPIC compilation error
dmd --version
DMD64 D Compiler v2.071.2
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright
on debian testing.
dub is installed via apt-get.
Should I revert to an earlier version? Or what?
The program:
importstd.stdio;
voidmain()
{//int[]t1;
//t1~=1;
//t1~=2;
//writeln ("t1 = ", t1);
}
fails with the 442 lines of error:
/usr/bin/ld: test.o: relocation R_X86_64_32 against symbol
`__dmd_personality_v0' can not be used when making a shared object;
recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_224_3b4.o): relocation
R_X86_64_32 against symbol `__dmd_personality_v0' can not be used when
making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_227_4a2.o): relocation
R_X86_64_32 against symbol `__dmd_personality_v0' can not be used when
making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_229_5cc.o): relocation
R_X86_64_32 against symbol `__dmd_personality_v0' can not be used when
making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_626_47b.o):
relocation R_X86_64_32 against symbol `_D6object9Throwable7__ClassZ' can
not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_628_776.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not be
used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(dwarfeh_62b_6b9.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not be
used when making a shared object; recompile with -fPIC
...
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(aaA_51a_53e.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not be
used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
--- errorlevel 1
Re: strange -fPIC compilation error
On 10/30/2016 04:03 PM, Lodovico Giaretta via Digitalmars-d-learn wrote:
On Sunday, 30 October 2016 at 18:02:28 UTC, Charles Hixson wrote:
dmd --version
DMD64 D Compiler v2.071.2
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright
on debian testing.
dub is installed via apt-get.
Should I revert to an earlier version? Or what?
The program:
importstd.stdio;
voidmain()
{//int[]t1;
//t1~=1;
//t1~=2;
//writeln ("t1 = ", t1);
}
fails with the 442 lines of error:
/usr/bin/ld: test.o: relocation R_X86_64_32 against symbol
`__dmd_personality_v0' can not be used when making a shared object;
recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_224_3b4.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_227_4a2.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_229_5cc.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_626_47b.o): relocation
R_X86_64_32 against symbol `_D6object9Throwable7__ClassZ' can not be
used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_628_776.o): relocation
R_X86_64_32 against symbol `__dmd_personality_v0' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(dwarfeh_62b_6b9.o): relocation
R_X86_64_32 against symbol `__dmd_personality_v0' can not be used
when making a shared object; recompile with -fPIC
...
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(aaA_51a_53e.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
--- errorlevel 1
Are you on Ubuntu 16.10, or some other system with an hardened
toolchain? If that's the case, you should compile with `-fPIC
-defaultlib=libphobos2.so`. You can put those options in your dmd.conf
configuration file, so that you don't have to type them every time.
Well, I'm using debian, but I've never had to do anything of that nature
before. OTOH, it's been a couple of months if this is a new change.
Re: strange -fPIC compilation error
Just as a test I tried it with ldc, and, as expected, there wasn't any
problem.
On 10/30/2016 11:02 AM, Charles Hixson via Digitalmars-d-learn wrote:
dmd --version
DMD64 D Compiler v2.071.2
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright
on debian testing.
dub is installed via apt-get.
Should I revert to an earlier version? Or what?
The program:
importstd.stdio;
voidmain()
{//int[]t1;
//t1~=1;
//t1~=2;
//writeln ("t1 = ", t1);
}
fails with the 442 lines of error:
/usr/bin/ld: test.o: relocation R_X86_64_32 against symbol
`__dmd_personality_v0' can not be used when making a shared object;
recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_224_3b4.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_227_4a2.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_229_5cc.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_626_47b.o):
relocation R_X86_64_32 against symbol `_D6object9Throwable7__ClassZ'
can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_628_776.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(dwarfeh_62b_6b9.o): relocation
R_X86_64_32 against symbol `__dmd_personality_v0' can not be used when
making a shared object; recompile with -fPIC
...
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(aaA_51a_53e.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
--- errorlevel 1
Re: strange -fPIC compilation error
So now I removed the repository version of dmd and dub, downloaded DMD64
D Compiler v2.072.0-b2, used dkpg to install it, and appear to get the
same errors.
(Well, actually I removed the commenting out of the code, but it
compiles and runs properly with ldc2.)
On 10/30/2016 11:02 AM, Charles Hixson via Digitalmars-d-learn wrote:
dmd --version
DMD64 D Compiler v2.071.2
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright
on debian testing.
dub is installed via apt-get.
Should I revert to an earlier version? Or what?
The program:
importstd.stdio;
voidmain()
{//int[]t1;
//t1~=1;
//t1~=2;
//writeln ("t1 = ", t1);
}
fails with the 442 lines of error:
/usr/bin/ld: test.o: relocation R_X86_64_32 against symbol
`__dmd_personality_v0' can not be used when making a shared object;
recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_224_3b4.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_227_4a2.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(exception_229_5cc.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_626_47b.o):
relocation R_X86_64_32 against symbol `_D6object9Throwable7__ClassZ'
can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(dmain2_628_776.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld:
/usr/lib/x86_64-linux-gnu/libphobos2.a(dwarfeh_62b_6b9.o): relocation
R_X86_64_32 against symbol `__dmd_personality_v0' can not be used when
making a shared object; recompile with -fPIC
...
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libphobos2.a(aaA_51a_53e.o):
relocation R_X86_64_32 against symbol `__dmd_personality_v0' can not
be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
--- errorlevel 1
Re: strange -fPIC compilation error
On 10/30/2016 05:14 PM, Lodovico Giaretta via Digitalmars-d-learn wrote:
On Monday, 31 October 2016 at 00:08:59 UTC, Charles Hixson wrote:
So now I removed the repository version of dmd and dub, downloaded
DMD64 D Compiler v2.072.0-b2, used dkpg to install it, and appear to
get the same errors.
(Well, actually I removed the commenting out of the code, but it
compiles and runs properly with ldc2.)
I don't think it's a problem of DMD. This is due to your gcc
installation being hardened, that is, configured to produce PIE by
default. To produce PIE, the linker needs to be fed PIC, which DMD
does not produce by default. The -fPIC flags makes DMD produce PIC out
of your sources. The problem is, libphobos2.a (the static version of
Phobos) is not compiled with -fPIC, so even if your code is PIC, gcc
will complain. The workaround is to use -defaultlib=libphobos2.so to
dynamically link with the shared version of Phobos. Being it a shared
object, it does not give any problem with PIE.
Looking on internet, I didn't find any clue about Debian shipping an
hardened gcc, but this is the only cause I can think of for the
behaviour you are experiencing.
Well, that certainly changed the error messages. With
dmd -defaultlib=/usr/lib/x86_64-linux-gnu/libphobos2.so test.d
I get:
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121): Error: found
'nothrow' when expecting '{'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1123): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1124): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1125): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1126): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1127): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1128): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1129): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1133): Error: asm
statements must end in ';'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1136): Error: found
'private' instead of statement
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1146): Error: no
identifier for declarator add
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error: no
identifier for declarator usDone
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error:
Declaration expected, not ':'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1157): Error:
Declaration expected, not '('
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not 'foreach'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not '0'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error: no
identifier for declarator __fhnd_info[fd]
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error:
Declaration expected, not '='
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1165): Error:
Declaration expected, not 'return'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1167): Error:
unrecognized declaration
/usr/include/dmd/phobos/std/typecons.d(1124): Error: semicolon expected
following function declaration
Re: strange -fPIC compilation error
On 10/30/2016 11:34 PM, Daniel Kozak via Digitalmars-d-learn wrote:
Dne 31.10.2016 v 02:30 Charles Hixson via Digitalmars-d-learn napsal(a):
Well, that certainly changed the error messages. With
dmd -defaultlib=/usr/lib/x86_64-linux-gnu/libphobos2.so test.d
I get:
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121): Error:
found 'nothrow' when expecting '{'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1123): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1124): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1125): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1126): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1127): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1128): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1129): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1133): Error: asm
statements must end in ';'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1136): Error:
found 'private' instead of statement
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1146): Error: no
identifier for declarator add
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error: no
identifier for declarator usDone
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error:
Declaration expected, not ':'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1157): Error:
Declaration expected, not '('
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not 'foreach'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not '0'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error: no
identifier for declarator __fhnd_info[fd]
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error:
Declaration expected, not '='
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1165): Error:
Declaration expected, not 'return'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1167): Error:
unrecognized declaration
/usr/include/dmd/phobos/std/typecons.d(1124): Error: semicolon
expected following function declaration
This seems to be problem with your installation, you probably have
diferen version of dmd compiler and phobos library. So you should
uninstall all your dmd packages and make sure there is no
/usr/include/dmd left in your system. And instal dmd only from one
source (d-apt idealy).
I've done that 2 or 3 times. If that's the problem, then there are
different versions stored in the repository. Since I'm on debian
testing I'd been assuming that there'd been some system change since I'd
last used the compiler, and the debs weren't yet up to date. The only
updates to my system prior to the compiler breaking HAD been via
apt-get. Since then I've used dpkg remove and install a couple of times
to try other versions of dmd with no benefit.
Currently dmd-bin version 2.071.2-0
libphobos2.071.2-0
libphobos2.071.2-0
so they're LISTED as being the same version. And dmd.conf was installed
by the deb, and is (eliminating the comments):
[Environment32]
DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import
-L-L/usr/lib/i386-linux-gnu -L--export-dynamic
[Environment64]
DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import
-L-L/usr/lib/x86_64-linux-gnu -L--export-dynamic
But somewhere during the process (which included the nightly system
update) the error messages changed, and now:
dmd test.d
yields:
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121): Error: found
'nothrow' when expecting '{'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1123): Error:
mismatched number of curly brackets
...
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1167): Error:
unrecognized declaration
/usr/include/dmd/phobos/std/typecons.d(1124): Error: semicolon expected
following function declaration
Re: strange -fPIC compilation error
On 10/31/2016 09:26 AM, Charles Hixson via Digitalmars-d-learn wrote:
On 10/30/2016 11:34 PM, Daniel Kozak via Digitalmars-d-learn wrote:
Dne 31.10.2016 v 02:30 Charles Hixson via Digitalmars-d-learn napsal(a):
Well, that certainly changed the error messages. With
dmd -defaultlib=/usr/lib/x86_64-linux-gnu/libphobos2.so test.d
I get:
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121): Error:
found 'nothrow' when expecting '{'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1123): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1124): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1125): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1126): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1127): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1128): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1129): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1133): Error: asm
statements must end in ';'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1136): Error:
found 'private' instead of statement
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1146): Error: no
identifier for declarator add
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error: no
identifier for declarator usDone
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error:
Declaration expected, not ':'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1157): Error:
Declaration expected, not '('
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not 'foreach'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not '0'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error: no
identifier for declarator __fhnd_info[fd]
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error:
Declaration expected, not '='
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1165): Error:
Declaration expected, not 'return'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1167): Error:
unrecognized declaration
/usr/include/dmd/phobos/std/typecons.d(1124): Error: semicolon
expected following function declaration
This seems to be problem with your installation, you probably have
diferen version of dmd compiler and phobos library. So you should
uninstall all your dmd packages and make sure there is no
/usr/include/dmd left in your system. And instal dmd only from one
source (d-apt idealy).
I've done that 2 or 3 times. If that's the problem, then there are
different versions stored in the repository. Since I'm on debian
testing I'd been assuming that there'd been some system change since
I'd last used the compiler, and the debs weren't yet up to date. The
only updates to my system prior to the compiler breaking HAD been via
apt-get. Since then I've used dpkg remove and install a couple of
times to try other versions of dmd with no benefit.
Currently dmd-bin version 2.071.2-0
libphobos2.071.2-0
libphobos2.071.2-0
so they're LISTED as being the same version. And dmd.conf was
installed by the deb, and is (eliminating the comments):
[Environment32]
DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import
-L-L/usr/lib/i386-linux-gnu -L--export-dynamic
[Environment64]
DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import
-L-L/usr/lib/x86_64-linux-gnu -L--export-dynamic
But somewhere during the process (which included the nightly system
update) the error messages changed, and now:
dmd test.d
yields:
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121): Error: found
'nothrow' when expecting '{'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1123): Error:
mismatched number of curly brackets
...
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1167): Error:
unrecognized declaration
/usr/include/dmd/phobos/std/typecons.d(1124): Error: semicolon
expected following function declaration
FWIW starting at /usr/include/dmd/druntime/import/core/stdc/stdio.d(1121)::
asm nothrow @nogc
{
mov EDX, num;
lock;
inc _iSemLockCtrs[EDX * 2];
so nothrow isn't being seen as appropriate at the beginning of an asm
block. After that I think it gets confused as 1123 doesn't HAVE a brace
(i.e. curly bracket) in it.
Re: strange -fPIC compilation error
On 10/31/2016 11:23 AM, Daniel Kozak via Digitalmars-d-learn wrote:
Dne 31.10.2016 v 18:06 Charles Hixson via Digitalmars-d-learn napsal(a):
On 10/31/2016 09:26 AM, Charles Hixson via Digitalmars-d-learn wrote:
On 10/30/2016 11:34 PM, Daniel Kozak via Digitalmars-d-learn wrote:
Dne 31.10.2016 v 02:30 Charles Hixson via Digitalmars-d-learn
napsal(a):
Well, that certainly changed the error messages. With
dmd -defaultlib=/usr/lib/x86_64-linux-gnu/libphobos2.so test.d
I get:
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121): Error:
found 'nothrow' when expecting '{'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1123): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1124): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1125): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1126): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1127): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1128): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1129): Error:
mismatched number of curly brackets
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1133): Error:
asm statements must end in ';'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1136): Error:
found 'private' instead of statement
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1146): Error:
no identifier for declarator add
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error:
no identifier for declarator usDone
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1149): Error:
Declaration expected, not ':'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1157): Error:
Declaration expected, not '('
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not 'foreach'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1159): Error:
Declaration expected, not '0'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error:
no identifier for declarator __fhnd_info[fd]
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1164): Error:
Declaration expected, not '='
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1165): Error:
Declaration expected, not 'return'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1167): Error:
unrecognized declaration
/usr/include/dmd/phobos/std/typecons.d(1124): Error: semicolon
expected following function declaration
This seems to be problem with your installation, you probably have
diferen version of dmd compiler and phobos library. So you should
uninstall all your dmd packages and make sure there is no
/usr/include/dmd left in your system. And instal dmd only from one
source (d-apt idealy).
I've done that 2 or 3 times. If that's the problem, then there are
different versions stored in the repository. Since I'm on debian
testing I'd been assuming that there'd been some system change since
I'd last used the compiler, and the debs weren't yet up to date. The
only updates to my system prior to the compiler breaking HAD been
via apt-get. Since then I've used dpkg remove and install a couple
of times to try other versions of dmd with no benefit.
Currently dmd-bin version 2.071.2-0
libphobos2.071.2-0
libphobos2.071.2-0
so they're LISTED as being the same version. And dmd.conf was
installed by the deb, and is (eliminating the comments):
[Environment32]
DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import
-L-L/usr/lib/i386-linux-gnu -L--export-dynamic
[Environment64]
DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import
-L-L/usr/lib/x86_64-linux-gnu -L--export-dynamic
But somewhere during the process (which included the nightly system
update) the error messages changed, and now:
dmd test.d
yields:
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121): Error:
found 'nothrow' when expecting '{'
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1123): Error:
mismatched number of curly brackets
...
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1167): Error:
unrecognized declaration
/usr/include/dmd/phobos/std/typecons.d(1124): Error: semicolon
expected following function declaration
FWIW starting at
/usr/include/dmd/druntime/import/core/stdc/stdio.d(1121)::
asm nothrow @nogc
{
mov EDX, num;
lock;
inc _iSemLockCtrs[EDX * 2];
so nothrow isn't being seen as appropriate at the beginning of an asm
block. After that I think it gets confused as 1123 doesn't HAVE a
brace (i.e. curly bracket) in it.
when you type dmd --version what it
Re: strange -fPIC compilation error
On 10/31/2016 12:31 PM, Daniel Kozak via Digitalmars-d-learn wrote: Dne 31.10.2016 v 20:20 Charles Hixson via Digitalmars-d-learn napsal(a): ... but dmd -defaultlib=libphobos2.so -fPIC test.d works. It shouldn't be required (as in the default /etc/dmd.conf should handle it correctly, but I can deal with it now. It should work, it is possible that you have some another dmd.conf somewhere? I did have a couple lying around, but they worked fine in the past, and renaming them didn't fix, or even just change, anything. I've still got some others on my backup partition, but I can't imagine that they would be in use. One of them was there because I had a few macros that were specified in an external ddoc file that was used by one project, e.g.
Re: strange -fPIC compilation error
On 11/01/2016 12:52 AM, Nordlöw via Digitalmars-d-learn wrote: On Tuesday, 1 November 2016 at 07:15:19 UTC, Mike Parker wrote: but dmd -defaultlib=libphobos2.so -fPIC test.d works. It shouldn't be required (as in the default /etc/dmd.conf should handle it correctly, but I can deal with it now. Can this be fed as parameters to the DMD call (programmatically) instead of manual editing of dmd.conf? I did feed it into the dmd call programmatically. Thus: dmd -defaultlib=libphobos2.so -fPIC test.d I'm reluctant to edit the dmd.conf file as I don't really understand the syntax...and I have to be root to do the editing.
Re: strange -fPIC compilation error
On 11/01/2016 10:34 AM, bachmeier via Digitalmars-d-learn wrote: On Tuesday, 1 November 2016 at 17:23:54 UTC, Charles Hixson wrote: On 11/01/2016 12:52 AM, Nordlöw via Digitalmars-d-learn wrote: On Tuesday, 1 November 2016 at 07:15:19 UTC, Mike Parker wrote: but dmd -defaultlib=libphobos2.so -fPIC test.d works. It shouldn't be required (as in the default /etc/dmd.conf should handle it correctly, but I can deal with it now. Can this be fed as parameters to the DMD call (programmatically) instead of manual editing of dmd.conf? I did feed it into the dmd call programmatically. Thus: dmd -defaultlib=libphobos2.so -fPIC test.d I'm reluctant to edit the dmd.conf file as I don't really understand the syntax...and I have to be root to do the editing. https://dlang.org/dmd-linux.html#dmd-conf You can copy dmd.conf to the current working directory and try out changes there. That will only affect the current program. Then you can move it to your home directory. I'll probably do that eventually...but for now I'm just pasting a comment at the start of each file that tells how to compile it, e.g.: //ldc2 -Dddocs -c msg.d<<--no documentation generated //dmd -Dddocs -c msg.d //rdmd -main -Dddocs -unittest -defaultlib=libphobos2.so -fPIC msg.d
Re: Sockets and using them...
On 11/05/2016 11:02 PM, Era Scarecrow via Digitalmars-d-learn wrote: So I've got a project where I want to create basically a decentralized chat program where every program is a host and a client. When you connect all connections can go through to route the chat to everyone else. So to make this work I've looked over the sockets package and I don't quite follow how you'd make it so it works a lot like a web browser, aka when you get a connection you redirect to a different port so you have 1 listening port for new connections so it can act as a server. What settings or configuration would I need to be able to do that? That sounds more like a job for udp than tcp sockets. You'd need to implement an ack/nak protocol because udp doesn't guarantee delivery, but unless you want a central server I don't see how you could use tcp. The central server wouldn't need to do much, and could delegate most of the processing, but it wouldn't be decentralized. I was looking into using tcp (actually zmq) for something similar awhile back and tcp just didn't seem to support actually decentralized communication. Udp did...which meant I couldn't use zmq. That was a real pity because zmq is basically a very nice package, and easy to wrap.
problem with isnan
The line: assert(isnan (c.curActivation), "cell has unexpected curActivation: %s".format(c.curActivation)); throws the exception: core.exception.AssertError@cell.d(285): cell has unexpected curActivation: nan and I've looked at it backwards and forwards and don't understand why. It's *supposed* to be nan, and the assert message reports that it is, but it should pass the assert test, not throw an assertion. What am I doing wrong?
Re: problem with isnan
On 11/10/2016 08:47 AM, Adam D. Ruppe via Digitalmars-d-learn wrote: On Thursday, 10 November 2016 at 16:41:56 UTC, Charles Hixson wrote: It's *supposed* to be nan, and the assert message reports that it is, but it should pass the assert test, not throw an assertion. What am I doing wrong? How did you set it? There are like billions of different NaNs. I'm not sure if isnan checks for all of them. (I'm also not sure that it doesn't, the docs don't specify.) you might try using std.math.isNaN instead and see what it does. It was default initialized by the class instance: classCell ... floatcurActivation; ... The this method doesn't have any mention of a few variables that are supposed to be default initialized, or which curActivation is one. I suppose I could set it to be -2.0 or something, but that doesn't really tell me what's going on.
Re: problem with isnan
On 11/11/2016 10:31 AM, pineapple via Digitalmars-d-learn wrote: On Thursday, 10 November 2016 at 16:47:30 UTC, Adam D. Ruppe wrote: On Thursday, 10 November 2016 at 16:41:56 UTC, Charles Hixson wrote: It's *supposed* to be nan, and the assert message reports that it is, but it should pass the assert test, not throw an assertion. What am I doing wrong? How did you set it? There are like billions of different NaNs. I'm not sure if isnan checks for all of them. (I'm also not sure that it doesn't, the docs don't specify.) you might try using std.math.isNaN instead and see what it does. Incidentally, I just recently submitted a PR to fix this. What probably happened is that you're referring to a limited `isnan` method defined as a unittest utility method in object.d that should have been private but wasn't. You want to use `isNan` instead. Thank you. Unfortunately: importstd.math; ... assert(isNan (c.curActivation), "cell has unexpected curActivation: %s".format(c.curActivation)); yields: cell.d(292): Error: undefined identifier 'isNan', did you mean overloadset 'isnan'? while: importstd.math; ... assert(std.math.isnan (c.curActivation), "cell has unexpected curActivation: %s".format(c.curActivation)); yields: core.exception.AssertError@cell.d(310): cell has unexpected idno: 636144943519357244 That is, indeed, an unexpected value, unless it's some representation of Nan, in which case it should have passed the assert. I notice that it doesn't appear to be a float, which puzzles me.
Re: problem with isnan
On 11/11/2016 01:34 PM, John C via Digitalmars-d-learn wrote: On Friday, 11 November 2016 at 20:55:52 UTC, Charles Hixson wrote: Thank you. Unfortunately: importstd.math; ... assert(isNan (c.curActivation), "cell has unexpected curActivation: %s".format(c.curActivation)); yields: cell.d(292): Error: undefined identifier 'isNan', did you mean overloadset 'isnan'? It should be isNaN. Ok, now it seems the same as std.math.isnan, (i.e., it works properly). On looking over the error messages more closely (just now) I noticed that the line number had now changed. Whoops! It just *LOOKED* like the error hadn't been fixed, where it had actually moved onto the next one. The hint should have been that it was printing an integer value...I mean besides the changed line number.
Re: Is it possible to store different generic types in ex. an
On 11/09/2016 07:46 AM, Is it possible to store different generic types?
via Digitalmars-d-learn wrote:
On Wednesday, 9 November 2016 at 15:44:59 UTC, Is it possible to store
different generic types? wrote:
Is it possible to store different generic types in ex. somekind of
container such as an array, hashtable etc.
Let's say we got
class Foo(T) {
...
}
Would it be possible to store something like
Foo[] foos; // Where Foo of course should allow any generic version
of Foo
Ex.
Foo!int and Foo!string should both be able to be stored inside the
array.
If so how would one construct an array like that or is there some
other container that may be able to do it?
I'm aware that I could have Foo inherit a base class and then use
casting, but I would like to avoid casting if possible.
You don't necessarily have to do casting if all of the objects that you
are going to store use the same methods for the purposes that you are
going to use them. Just craft an interface that they all implement and
inherit from that. But if they don't all implement all the methods you
are going to need, you can't avoid casting unless you do something like:
1) create a dummy class (I don't *think* an abstract class would work)
that implements all the methods you are going to need with dummy methods.
2) inherit all the classes you are going to use from that class
3) be sure to use override on the methods that are actually implemented.
Even then I'm not sure this would work. Since you are doing something like:
Dummy[Key] stuff;
when you retrieve an item from stuff, what you retrieve will be the
actual item, but the system will probably think it's a Dummy unless you
properly cast it. So go with the interface approach, even if that means
implementing dummy methods for some of the classes. (OTOH, I haven't
actually *tried* that inherit from a common class...it might work. I'd
just bet against it without casting.)
The basic idea here is that all the objects that you store have to
implement all the methods that you are going to use on any of them, but
the implementations can be something like:
void method1(){assert (false, "You should never come here"); }
shared arrray problem
I have a piece of code that looks thus:
/**Returns an editable file header missing the header length and data
* length portions. Those cannot be edited by a routine outside this
class.
* Access to them is available via the lenHead and lenRec functions.
* Warning: Do NOT change the size of the header. If you do the size
* will be reset to the current value before it is saved, and results are
* unpredictable. Also do not replace it. This class maintains it's own
* pointer to the header, and your replacement will be ignored. */
ubyte[]header()@property{return fHead[4..$];}
I want other classes to be able to modify the tail of the array, but not
to resize it. Eventually it should be written to a file. This way works
(well, should work) but looks radically unsafe for the reasons indicated
in the comments. I'm sure there must be a better way, but can't think
of what it would be. The only alternative I've been able to think of
is, approx:
boolsaveToHeader (ubyte[] rec){ ... buf[4..$] = rec[0..$]; ... }
but that would be guaranteed to have an extra copying step, and it's
bool because if the length of the passed parameter isn't correct
saveToHeader would fail. It may still be better since in this case the
copying would be rather minimal, but the general problem bothers me
because I don't see a good solution.
Re: fPIC Error
On 11/18/2016 10:35 PM, deadalnix via Digitalmars-d-learn wrote: On Thursday, 3 November 2016 at 06:11:48 UTC, rikki cattermole wrote: [Environment32] DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import -L-L/usr/lib/i386-linux-gnu -L--export-dynamic -fPIC -defaultlib=libphobos2.so [Environment64] DFLAGS=-I/usr/include/dmd/phobos -I/usr/include/dmd/druntime/import -L-L/usr/lib/x86_64-linux-gnu -L--export-dynamic -fPIC -defaultlib=libphobos2.so I'm sorry bit that's horseshit. I'm not compiling everything with fPIC because DMD can't get it's act straight. IIRC, LDC didn't have that problem. I don't remember testing gdc. But, yes, it is quite annoying.
Re: shared arrray problem
On 11/19/2016 11:10 AM, Nicolas Gurrola via Digitalmars-d-learn wrote:
On Saturday, 19 November 2016 at 18:51:05 UTC, Charles Hixson wrote:
ubyte[]header()@property {return fHead[4..$];}
This method should do what you want. You are only returning a slice of
the fHead array, so if the caller modifies the length it will only
affect of the return value, and not the length of fHead itself.
It's worse than that, if they modify the length the array may be
reallocated in RAM so that the pointers held by the containing class do
not point to the changed values. (Read the header comments...it's not
nice at all.)
More, the class explicitly requires the array to be a particular length
as it needs to fit into a spot in a binary file, so I really want to
forbid any replacement of the array for any reason. The first four
bytes are managed by the class and returned via alternate routines which
do not allow the original values to be altered, and this is necessary.
I could make them immutable, but actually what I do is return, e.g., an
int of 256 * fHead[0] + fHead[1], which is the length of the header.
It's an int to allow negative values to be returned in case of error.
So what I'll probably eventually decide on is some variation of
saveToHead, e.g.:
boolsaveToHeader(ubyte[] rec)
{ if(rec.length + 4 > dheader)returnfalse;
fHead[4..recLen + 4] = rec[0..$];
returntrue;
}
unless I can think of something better. Actually for this particular
case that's not a bad approach, but for the larger problem it's a lousy
kludge.
Re: shared arrray problem
On 11/19/2016 01:50 PM, ag0aep6g via Digitalmars-d-learn wrote: On 11/19/2016 10:26 PM, Charles Hixson via Digitalmars-d-learn wrote: It's worse than that, if they modify the length the array may be reallocated in RAM so that the pointers held by the containing class do not point to the changed values. (Read the header comments...it's not nice at all.) Arguably, any D programmer must be aware that appending to a dynamic array potentially means making a copy of the data, and that changes to length are not visible to other views of the data. But it's an opportunity to mess up, for sure. You could return a wrapper around the array that supports editing the data but not changing the length or appending. Looks like std.experimental.typecons.Final [1] is supposed to be that wrapper. But in a little test I can still set the length. Not sure if that's a bug, or if Final has slightly different goals. [1] https://dlang.org/phobos/std_experimental_typecons.html#.Final Yes. I was hoping someone would pop up with some syntax making the array, but not its contents, const or immutable, which I couldn't figure out how to do, and which is what I really hoped would be the answer, but it appears that this isn't part of the syntax. If the array is constant, so is it's contents. I really *can't* allow the length to be changed, and if the array is reallocated, it won't get saved. But the contents of the array are intended to be changed by the calling routines. Again, for this particular problem the kludge of copying the values into the array works fine (and is what I've decided to do), but that's not a good general solution to this kind of problem.
Re: shared arrray problem
On 11/19/2016 05:52 PM, ag0aep6g via Digitalmars-d-learn wrote: On 11/20/2016 01:33 AM, Charles Hixson via Digitalmars-d-learn wrote: Yes. I was hoping someone would pop up with some syntax making the array, but not its contents, const or immutable, which I couldn't figure out how to do, and which is what I really hoped would be the answer, but it appears that this isn't part of the syntax. Yup, head const is not part of the language. You'd have to find a library solution or write something yourself. I really *can't* allow the length to be changed, Your emphasis suggests that user could break things for your code. They can't. Any changes to the length will only affect the slice on the user's end. They can only fool themselves. That may be bad enough to warrant a more restricted return type, but for your code it's safe to return a plain dynamic array. Whether you would call the change "break things for your code" might be dubious. It would be effectively broken, even if technically my code was doing the correct thing. But my code wouldn't be storing the data that needed storing, so effectively it would be broken. "Write something for yourself" is what I'd like to do, given that the language doesn't have that built-in support, but I can't see how to do it. I want to end up with a continuous array of ubytes of a given length with certain parts reserved to only be directly accessible to the defining class, and other parts accessible to the calling class(es). And the length of the array isn't known until run time. So I guess the only safe solution is to do an extra copy...which isn't a problem in this particular application as I only need to do it twice per file opening (once on opening, once on closing), but for other applications would be a real drag.
Re: shared arrray problem
On 11/20/2016 03:42 AM, ag0aep6g via Digitalmars-d-learn wrote:
On 11/20/2016 04:34 AM, Charles Hixson via Digitalmars-d-learn wrote:
Whether you would call the change "break things for your code" might be
dubious. It would be effectively broken, even if technically my code
was doing the correct thing. But my code wouldn't be storing the data
that needed storing, so effectively it would be broken.
I don't see how it's dubious. It's an error by the user. When users
are given a dynamic array (and not by reference), they cannot expect
that your code sees changes to length. That's just not how arrays
work. When a user has that wrong expectation, and writes wrong code
because of it, then it's arguably their own fault. However, if you
want you can hold their hand a bit and make the mistake less likely.
"Write something
for yourself" is what I'd like to do, given that the language doesn't
have that built-in support, but I can't see how to do it.
Wrap the array in a struct that has indexing, but doesn't allow
setting the length or appending. Here's a quick prototype:
struct ConstLengthArray(E)
{
private E[] data;
this(E[] arr) { this.data = arr; }
ref inout(E) opIndex(size_t i) inout { return data[i]; }
@property size_t length() const { return data.length; }
}
void main()
{
auto cla = ConstLengthArray!ubyte([1, 2, 3, 4, 5]);
/* Mutating elements is allowed: */
cla[0] = 10;
assert(cla[0] == 10);
/* No setting length, no appending: */
static assert(!__traits(compiles, cla.length = 3));
static assert(!__traits(compiles, cla ~= 6));
}
You might want to add support for slicing, concatenation, etc. Maybe
allow implicit conversion to const(E[]), though that would also allow
conversion to const(E)[] and that has a settable length again.
Well, that precise approach wouldn't work. (The traits aren't a part of
the sturct, e.g.), but returning a struct (or perhaps a class) rather
than an actual array has promise. It could even allow separate callers
to have separate views of the data based on some sort of registered key,
which they could share on an as-needed basis. That's too much overhead
work for this project, but has promise for the more general problem.
Re: shared arrray problem
On 11/20/2016 03:42 AM, ag0aep6g via Digitalmars-d-learn wrote:
On 11/20/2016 04:34 AM, Charles Hixson via Digitalmars-d-learn wrote:
Whether you would call the change "break things for your code" might be
dubious. It would be effectively broken, even if technically my code
was doing the correct thing. But my code wouldn't be storing the data
that needed storing, so effectively it would be broken.
I don't see how it's dubious. It's an error by the user. When users
are given a dynamic array (and not by reference), they cannot expect
that your code sees changes to length. That's just not how arrays
work. When a user has that wrong expectation, and writes wrong code
because of it, then it's arguably their own fault. However, if you
want you can hold their hand a bit and make the mistake less likely.
"Write something
for yourself" is what I'd like to do, given that the language doesn't
have that built-in support, but I can't see how to do it.
Wrap the array in a struct that has indexing, but doesn't allow
setting the length or appending. Here's a quick prototype:
struct ConstLengthArray(E)
{
private E[] data;
this(E[] arr) { this.data = arr; }
ref inout(E) opIndex(size_t i) inout { return data[i]; }
@property size_t length() const { return data.length; }
}
void main()
{
auto cla = ConstLengthArray!ubyte([1, 2, 3, 4, 5]);
/* Mutating elements is allowed: */
cla[0] = 10;
assert(cla[0] == 10);
/* No setting length, no appending: */
static assert(!__traits(compiles, cla.length = 3));
static assert(!__traits(compiles, cla ~= 6));
}
You might want to add support for slicing, concatenation, etc. Maybe
allow implicit conversion to const(E[]), though that would also allow
conversion to const(E)[] and that has a settable length again
Thinking it over a bit more, the item returned would need to be a
struct, but the struct wouldn't contain the array, it would just contain
a reference to the array and a start and end offset. The array would
need to live somewhere else, in the class (or struct...but class is
better as you don't want the array evaporating by accident) that created
the returned value. This means you are dealing with multiple levels of
indirection, so it's costly compared to array access, but cheap compared
to lots of large copies. So the returned value would be something like:
struct
{
private:
/** this is a reference to the data that lives elsewhere. It
should be a pointer, but I don't like the syntax*/
ubyte[] data;
intstart, end;///first and last valid indicies into data
public:
this (ubyte[] data, int start, int end)
{this.data = data; this.start = start; this.end = end;}
...
// various routines to access the data, but to limit the access to
the spec'd range, and
// nothing to change the bounds
}
Which is really the answer you already posted, but just a bit more
detail on the construct, and what it meant. (Yeah, I could allow types
other than ubyte as the base case, but I don't want to. I'm thinking of
this mainly as a means of sharing a buffer between applications where
different parts have exclusive access to different parts of the buffer,
and where the buffer will be written to a file with a single fwrite, or
since the underlying storage will be an array, it could even be
rawwrite). I don't want to specify any more than I must about how the
methods calling this will format the storage, and this means that those
with access to different parts may well use different collections of
types, but all types eventually map down to ubytes (or bytes), so ubytes
is the common ground. Perhaps I'll need to write inbuffer,outbuffer
methods/wrappings, but that's far in the future.
P.S.: The traits that I mentioned previously were those given by:
static assert(!__traits(compiles, cla.length = 3));
static assert(!__traits(compiles, cla ~= 6));
in your main routine. I assumed that they were validity tests. I don't
understand why they were static. I've never happened to use static
asserts, but I would assume that when they ran cla wouldn't be defined.
N.B.: Even this much is just thinking about design, not something I'll
actually do at the moment. But this is a problem I keep coming up
against, so a bit of thought now seemed a good idea.
Re: shared arrray problem
On 11/20/2016 12:41 PM, ag0aep6g via Digitalmars-d-learn wrote:
On 11/20/2016 09:09 PM, Charles Hixson via Digitalmars-d-learn wrote:
Thinking it over a bit more, the item returned would need to be a
struct, but the struct wouldn't contain the array, it would just contain
a reference to the array and a start and end offset. The array would
need to live somewhere else, in the class (or struct...but class is
better as you don't want the array evaporating by accident) that created
the returned value. This means you are dealing with multiple levels of
indirection, so it's costly compared to array access, but cheap compared
to lots of large copies. So the returned value would be something like:
struct
{
private:
/** this is a reference to the data that lives elsewhere. It should
be a pointer, but I don't like the syntax*/
ubyte[] data;
intstart, end;///first and last valid indicies into data
public:
this (ubyte[] data, int start, int end)
{this.data = data; this.start = start; this.end = end;}
...
// various routines to access the data, but to limit the access to
the spec'd range, and
// nothing to change the bounds
}
Instead of extra 'start' and 'end' fields you can slice the array. A
dynamic array already is just a reference coupled with a length, i.e.
a pointer with restricted indexing. So you can slice the original
array with your offsets and create the struct with that slice.
I feel like there is a misunderstanding somewhere, but I'm not sure on
whose side. As far as I can tell, your understanding of dynamic arrays
may be lacking, or maybe I don't understand what you're getting at.
While you are definitely correct about slices, I really feel more
comfortable with the start and end fields. I keep being afraid some
smart optimizer is going to decide I don't really need the entire
array. This is probably unjustified, but I find start and end fields
easier to think about. I don't think either of us misunderstands what's
going on, we just feel differently about methods that are approximately
equivalent. Slices would probably be marginally more efficient (well,
certainly so as you'd need two less fields), so if this were a public
library there would be an excellent argument for doing it your way. I
keep trying to think of a reason that start and end fields are better,
and failing. All I've got is that I feel more comfortable with them.
Which is really the answer you already posted, but just a bit more
detail on the construct, and what it meant. (Yeah, I could allow types
other than ubyte as the base case, but I don't want to. I'm thinking of
this mainly as a means of sharing a buffer between applications where
different parts have exclusive access to different parts of the buffer,
and where the buffer will be written to a file with a single fwrite, or
since the underlying storage will be an array, it could even be
rawwrite). I don't want to specify any more than I must about how the
methods calling this will format the storage, and this means that those
with access to different parts may well use different collections of
types, but all types eventually map down to ubytes (or bytes), so ubytes
is the common ground. Perhaps I'll need to write inbuffer,outbuffer
methods/wrappings, but that's far in the future.
Sure, go with a specialized type instead of a template, if that makes
more sense for your use case. As far as I see, the concept is
independent of the element type, so it seemed natural to make it a
template, but a special type is perfectly fine and probably has less
pitfalls.
P.S.: The traits that I mentioned previously were those given by:
static assert(!__traits(compiles, cla.length = 3));
static assert(!__traits(compiles, cla ~= 6));
in your main routine. I assumed that they were validity tests. I don't
understand why they were static. I've never happened to use static
asserts, but I would assume that when they ran cla wouldn't be defined.
Those are tests to ensure that cla's length cannot be set and that it
cannot be appended to. The asserts check that the code does not
compile, i.e. that you cannot do those things. They're static simply
because they can be. The code inside is not executed at run-time.
std.concurrency bug?
Is it a bug that an immutable struct cannot be sent to a thread? (It compiles without problem if I make all elements mutable.)
Re: std.concurrency bug?
On Tuesday, May 20, 2014 11:42:48 AM Ali Çehreli via Digitalmars-d-learn
wrote:
> On 05/20/2014 11:38 AM, Charles Hixson via Digitalmars-d-learn wrote:
> > Is it a bug that an immutable struct cannot be sent to a thread? (It
> > compiles without problem if I make all elements mutable.)
>
> Does the struct have any mutable indirection? Then it is illegal.
> Otherwise, can you demonstrate with minimal code please?
>
> Ali
Nope. Here it is (with the immutable tags removed):
/** This is the only message that one cell sends to another.*/
struct Msg
{ /** The cell id# of the sender of the message. */
uint64_tfrom;
/** The cell id# of the recipient of the message. */
uint64_tto;
/** The kind of action the message is impelling.*/
Act act;
/** The tick on which the message was accepted for transmission.
* This is set by std.datetime.Clock.currStdTime() */
longtick;
/** Distance between cells. Not currently well defined except in
* the case of two words, in which case it is the number of words of
* separation, where adjacent is 1. */
float dist;
/** Not currently well defined. */
int value;
this(uint64_t from, uint64_t to, Act act, float dist, int value)
{ this.from = from;
this.to = to;
this.act= act;
this.tick = Clock.currStdTime;
this.dist = dist;
this.value = value;
}
}
Re: std.concurrency bug?
On Tuesday, May 20, 2014 11:42:48 AM Ali Çehreli via Digitalmars-d-learn
wrote:
> On 05/20/2014 11:38 AM, Charles Hixson via Digitalmars-d-learn wrote:
> > Is it a bug that an immutable struct cannot be sent to a thread? (It
> > compiles without problem if I make all elements mutable.)
>
> Does the struct have any mutable indirection? Then it is illegal.
> Otherwise, can you demonstrate with minimal code please?
>
> Ali
Nearly a minimal example. If "voidsendMsg (Msg m){...}" is removed
there's
not compilation error.
import std.array;
import std.concurrency;
import std.datetime;
import std.format;
import std.stdio;
import std.stdint;
import std.string;
import std.variant;
enumnumCThreads = 4;
class UnexpectedMessage : Exception { this (string s) { super
(s);
} }
enumAct {create, activate, deactivate, wait, read, reset, save, done}
string toString(E)(E value) if (is(E == enum))
{ foreach (s; __traits(allMembers, E))
{ if (value == mixin("E." ~ s) ) return s; }
return null;
} // string toString (... for enum
/** This is the only message that one cell sends to another.*/
struct Msg
{ /** The cell id# of the sender of the message. */
immutable uint64_tfrom;
/** The cell id# of the recipient of the message. */
immutable uint64_tto;
/** The kind of action the message is impelling.*/
immutable Act act;
/** The tick on which the message was accepted for transmission.
* This is set by std.datetime.Clock.currStdTime() */
immutable longtick;
/** Distance between cells. Not currently well defined except in
* the case of two words, in which case it is the number of words of
* separation, where adjacent is 1. */
immutable float dist;
/** Not currently well defined. */
immutable int value;
this(uint64_t from, uint64_t to, Act act, float dist, int value)
{ this.from = from;
this.to = to;
this.act= act;
this.tick = Clock.currStdTime;
this.dist = dist;
this.value = value;
}
}
voidcellThread(size_t ndx, shared(Tid)[] cThreads)
{ /** Unprocessed message queue, indexed by cell id#. */
Msg msgs[uint64_t][];
booldone= false;
//TODO load the cells into the thread
/** Receive all messages in the mailbox. Wait 2 ns for response.
*/
while (!done && receiveTimeout (0.seconds,
(Msg m)
{ if (m.to in msgs)
{ msgs[m.to] ~= m;
}
else
{ msgs[m.to] = [m];
}
},
(Act act)
{ switch (act)
{ caseAct.done: //
end cell processing
done= true;
break;
default:
auto s = format (
"Error in
thread %s: received message
Act.%s",
ndx, act);
writefln (s);
throw new
UnexpectedMessage(s);
} // switch (act)
} //(Act act)
) )
{
} // while (!done && receiveTimeout
voidsendMsg (Msg m)
{ assert (m.to > 0);
assert (m.to < lastId);
int ndx = m.to % numCThreads;
Tid ct = cast(Tid)cThreads[ndx];
ct.send(m);
}
} // voidcellThread()
voidmain()
{ autocellTids= new shared(Tid)[numCThreads];
foreach (id; 0 .. numCThreads)
{ autocThread = spawn(&cellThread, id, cellTids);
cellTids[id]= cast(shared(Tid))cThread;
}
foreach (cThread; cellTids)
{ Tid ct = cast(Tid)cThread;
ct.send(Act.done);
}
}
Re: std.concurrency bug?
On Wednesday, May 21, 2014 01:19:32 PM Ali Çehreli via Digitalmars-d-learn
wrote:
> On 05/20/2014 05:24 PM, Charles Hixson via Digitalmars-d-learn wrote:
> > On Tuesday, May 20, 2014 11:42:48 AM Ali Çehreli via Digitalmars-d-learn
> >
> > wrote:
> >> On 05/20/2014 11:38 AM, Charles Hixson via Digitalmars-d-learn wrote:
> >>> Is it a bug that an immutable struct cannot be sent to a thread? (It
> >>> compiles without problem if I make all elements mutable.)
>
> Further reduced:
>
> import std.concurrency;
>
> struct S
> {
> immutable int i;
> }
>
> void foo()
> {}
>
> void main()
> {
> auto f = spawn(&foo);
>
> auto s = S();
> f.send(s);
> }
>
> I think this is a known issue with immutable and Variant, which
> std.concurrency uses for unknown messages. This looks related:
>
>https://issues.dlang.org/show_bug.cgi?id=5538
>
> Ali
Thanks. Fortunately, I don't really need for messages to be immutable, I just
thought it would be safer (as in avoiding the possibility of making a
mistake). They're stucts, and I never pass a pointer, just the struct. Being
immutable would allow me to guarantee that certain logic errors couldn't be
committed is all.
How to convert from ubyte[] to and from float?
What is the best way to convert from a part of a ubyte[] to a float? I've tried converting the ubyte[] into a uint, but neither casting the uint to a float nor to!float work. I suppose I could use a "trick record" union, but that seems inelegant. If I use pointers, the alignment may (unpredictably) not be proper (whatever that means these days).
parser bug?
The code:
voidgo(ulongrecNum)
{assert (buf[0] == (fh.sizeof + recNo * buf.length) & 0x7f);
if(dirty)
yields the error message:
ells$ dmd test.d
test.d(78): Error: buf[0] == fh.sizeof + recNo * buf.length must be
parenthesized when next to operator &
And I'll swear it looks parenthesized to me.
voidgo(ulongrecNum)
{ubytetst=(fh.sizeof + recNo * buf.length) & 0x7f;
assert (buf[0] == tst);
if(dirty)
compiles without error.
Any idea what's going on?
repack ubyte[] to use only 7 bits
Is there a standard way to do this? The code below is untested, as I
haven't yet written the x7to8 routine, and came up with a better way to
do what this was to accomplish, but it feels as if this should be
somewhere in the standard library, if I could only find it.
/** Repack the data from an array of ubytes into an array of ubytes of
* which only the last 7 are significant. The high bit will be set only
* if the byte would otherwise be zero.*/
byte[]x8to7 (ubyte[] bin)
{
ubyte[] bout;
//bit masks:0 => 0xfe = 1110, 0x00 =
//1 => 0x7f = 0111, 0x00 =
//2 => 0x3f = 0011, 0x80 = 1000
//3 => 0x1f = 0001, 0xc0 = 1100
//4 => 0x0f = , 0xe0 = 1110
//5 => 0x07 = 0111, 0xf0 =
//6 => 0x03 = 0011, 0xf8 = 1000
//7 => 0x01 = 0001, 0xfc = 1100
if (bin.length < 1)returnbout;
intfByte, fBit;
while(fByte < bin.length)
{if (fByte + 1 == bin.length && fBit > 1) break;
ubyteb;
switch (fBit)
{case0:
b=bin[fByte]/ 2;
break;
case1:
b=bin[fByte] & 0x7f;
break;
case2:
ubyteb1=(bin[fByte] & 0x3f) << 1;
ubyteb2=(bin[fByte + 1] & 0x80) >>> 7;
b~=(b1 | b2);
break;
case3:
ubyteb1=(bin[fByte] & 0x1f) << 2;
ubyteb2=(bin[fByte + 1] & 0xc0) >>> 6;
b~= (b1 | b2);
break;
case4:
ubyteb1=(bin[fByte] & 0x0f) << 3;
ubyteb2=(bin[fByte + 1] & 0xe0) >>> 5;
b~= (b1 | b2);
break;
case5:
ubyteb1=(bin[fByte] & 0x07) << 4;
ubyteb2=(bin[fByte + 1] & 0xf0) >>> 4;
b~= (b1 | b2);
break;
case6:
ubyteb1=(bin[fByte] & 0x03) << 5;
ubyteb2=(bin[fByte + 1] & 0xf8) >>> 3;
b~= (b1 | b2);
break;
case7:
ubyteb1=(bin[fByte] & 0x01) << 6;
ubyteb2=(bin[fByte + 1] & 0xfc) >>> 2;
b~= (b1 | b2);
break;
default:
assert (false, "This path should never be taken");
}//switch (fBit)
if(b == 0)bout~=0x80;
elsebout~=b;
fBit=fBit + 7;
if(fBit > 7)
{fByte++;
fBit -=7;
}
}
}
Re: repack ubyte[] to use only 7 bits
Your comments would be reasonable if this were destined for a library,
but I haven't even finished checking it (and probably won't since I've
switched to a simple zero elimination scheme). But this is a bit
specialized for a library...a library should probably deal with
arbitrary ints from 8 to 64 or 128 [but I don't think that the 128 bit
type is yet standard, only reserved]. I just thought that something
like that should be available, possibly along the lines of Python's pack
and unpack, and wondered where it was and what it was called.)
Additionally, I'm clearly not the best person to write the library
version, as I still have LOTS of trouble with D templates. And I have
not successfully wrapped my mind around D ranges...which is odd, because
neither Ruby nor Python ranges give me much trouble. Perhaps its the syntax.
As for " pure", "@safe", and "nothrow" ... I'd like to understand that I
COULD use those annotations. (The "in" I agree should be applied. I
understand that one.)
As for size_t for indexes...here I think we disagree. It would be a bad
mistake to use an index that size. I even considered using short or
ushort, but I ran into a comment awhile back saying that one should
never use those for local variables. This *can't* be an efficient
enough way that it would be appropriate to use it for a huge array...but
that should probably be documented if it were for a library. (If I were
to use size_t indexing, I'd want to modify things so that I could feed
it a file as input, and that's taking it well away from what I was
building it for: converting input to a redis database so that I could
feed it raw serial data streams without first converting it into human
readable formats. I wanted to make everything ASCII-7 binary data,
which, when I thought about it more, was overkill. All I need to do is
eliminate internal zeros, since C handles various extended character
formats by ignoring them.
I'm not clear what you mean by a "final switch". fBit must adopt
various different values during execution. If you mean it's the same as
a nest of if...else if ... statements, that's all I was really
expecting, but I thought switch was a bit more readable.
Binary literals would be more self-documenting, but would make the code
harder to read. If I'd though of them I might have used them...but
maybe not.
Output range? Here I'm not sure what you're suggesting, probably
because I don't understand D ranges.
The formatting got a bit messed up during pasting from the editor to the
mail message. I should have looked at it more carefully. My standard
says that unless the entire block is on a single line, the closing brace
should align with the opening brace. And I use tabs for spacing which
works quite well in the editor, but I *do* need to remember to convert
it to spaces before doing a cut and paste.
Thanks for your comments. I guess that means that there *isn't* a
standard function that does this.
Charles
On 12/06/2014 03:01 PM, bearophile via Digitalmars-d-learn wrote:
Charles Hixson:
byte[]x8to7 (ubyte[] bin)
Better to add some annotations, like pure, @safe, nothrow, if you can,
and to annotate the bin with an "in".
intfByte, fBit;
It's probably better to define them as size_t.
switch (fBit)
I think D doesn't yet allow this switch to be _meaningfully_ a final
switch.
b=bin[fByte] & 0x7f;
D allows binary number literals as 0b100110010101.
b~=(b1 | b2);
Perhaps an output range is better?
if(b == 0)bout~= 0x80;
elsebout~=b;
fBit=fBit + 7;
if(fBit > 7)
{fByte++;
fBit -=7;
The formatting seems a bit messy.
Bye,
bearophile
repack ubyte[] to use only 7 bits
Is there a standard way to do this? The code below is untested, as I
haven't yet written the x7to8 routine, and came up with a better way to
do what this was to accomplish, but it feels as if this should be
somewhere in the standard library, if I could only find it.
/** Repack the data from an array of ubytes into an array of ubytes of
* which only the last 7 are significant. The high bit will be set only
* if the byte would otherwise be zero.*/
byte[]x8to7 (ubyte[] bin)
{
ubyte[] bout;
//bit masks:0 => 0xfe = 1110, 0x00 =
//1 => 0x7f = 0111, 0x00 =
//2 => 0x3f = 0011, 0x80 = 1000
//3 => 0x1f = 0001, 0xc0 = 1100
//4 => 0x0f = , 0xe0 = 1110
//5 => 0x07 = 0111, 0xf0 =
//6 => 0x03 = 0011, 0xf8 = 1000
//7 => 0x01 = 0001, 0xfc = 1100
if (bin.length < 1)returnbout;
intfByte, fBit;
while(fByte < bin.length)
{if (fByte + 1 == bin.length && fBit > 1) break;
ubyteb;
switch (fBit)
{case0:
b=bin[fByte]/ 2;
break;
case1:
b=bin[fByte] & 0x7f;
break;
case2:
ubyteb1=(bin[fByte] & 0x3f) << 1;
ubyteb2=(bin[fByte + 1] & 0x80) >>> 7;
b~=(b1 | b2);
break;
case3:
ubyteb1=(bin[fByte] & 0x1f) << 2;
ubyteb2=(bin[fByte + 1] & 0xc0) >>> 6;
b~= (b1 | b2);
break;
case4:
ubyteb1=(bin[fByte] & 0x0f) << 3;
ubyteb2=(bin[fByte + 1] & 0xe0) >>> 5;
b~= (b1 | b2);
break;
case5:
ubyteb1=(bin[fByte] & 0x07) << 4;
ubyteb2=(bin[fByte + 1] & 0xf0) >>> 4;
b~= (b1 | b2);
break;
case6:
ubyteb1=(bin[fByte] & 0x03) << 5;
ubyteb2=(bin[fByte + 1] & 0xf8) >>> 3;
b~= (b1 | b2);
break;
case7:
ubyteb1=(bin[fByte] & 0x01) << 6;
ubyteb2=(bin[fByte + 1] & 0xfc) >>> 2;
b~= (b1 | b2);
break;
default:
assert (false, "This path should never be taken");
}//switch (fBit)
if(b == 0)bout~=0x80;
elsebout~=b;
fBit=fBit + 7;
if(fBit > 7)
{fByte++;
fBit -=7;
}
}
}
Re: repack ubyte[] to use only 7 bits
On 12/13/2014 03:20 AM, Manolo via Digitalmars-d-learn wrote:
On Saturday, 13 December 2014 at 10:09:27 UTC, Charles Hixson via
Digitalmars-d-learn wrote:
Is there a standard way to do this? The code below is untested, as I
haven't yet written the x7to8 routine, and came up with a better way
to do what this was to accomplish, but it feels as if this should be
somewhere in the standard library, if I could only find it.
/** Repack the data from an array of ubytes into an array of ubytes of
* which only the last 7 are significant. The high bit will be set only
* if the byte would otherwise be zero.*/
byte[]x8to7 (ubyte[] bin)
{
ubyte[] bout;
//bit masks:0 => 0xfe = 1110, 0x00 =
//1 => 0x7f = 0111, 0x00 =
//2 => 0x3f = 0011, 0x80 = 1000
//3 => 0x1f = 0001, 0xc0 = 1100
//4 => 0x0f = , 0xe0 = 1110
//5 => 0x07 = 0111, 0xf0 =
//6 => 0x03 = 0011, 0xf8 = 1000
//7 => 0x01 = 0001, 0xfc = 1100
if (bin.length < 1)returnbout;
intfByte, fBit;
while(fByte < bin.length)
{if (fByte + 1 == bin.length && fBit > 1) break;
ubyteb;
switch (fBit)
{case0:
b=bin[fByte]/ 2;
break;
case1:
b=bin[fByte] & 0x7f;
break;
case2:
ubyteb1=(bin[fByte] & 0x3f) << 1;
ubyteb2=(bin[fByte + 1] & 0x80) >>> 7;
b~=(b1 | b2);
break;
case3:
ubyteb1=(bin[fByte] & 0x1f) << 2;
ubyteb2=(bin[fByte + 1] & 0xc0) >>> 6;
b~= (b1 | b2);
break;
case4:
ubyteb1=(bin[fByte] & 0x0f) << 3;
ubyteb2=(bin[fByte + 1] & 0xe0) >>> 5;
b~= (b1 | b2);
break;
case5:
ubyteb1=(bin[fByte] & 0x07) << 4;
ubyteb2=(bin[fByte + 1] & 0xf0) >>> 4;
b~= (b1 | b2);
break;
case6:
ubyteb1=(bin[fByte] & 0x03) << 5;
ubyteb2=(bin[fByte + 1] & 0xf8) >>> 3;
b~= (b1 | b2);
break;
case7:
ubyteb1=(bin[fByte] & 0x01) << 6;
ubyteb2=(bin[fByte + 1] & 0xfc) >>> 2;
b~= (b1 | b2);
break;
default:
assert (false, "This path should never be taken");
}//switch (fBit)
if(b == 0)bout~=0x80;
elsebout~=b;
fBit=fBit + 7;
if(fBit > 7)
{fByte++;
fBit -=7;
}
}
}
Are you trying to make a "kind-of" Variable-Length quantity encoder ?
eg:
0b10101110: last bit not set, integrate 0b10101110 and stop reading.
0b10011001: last bit set, integrate 0b10011000 and continue to next byte.
http://en.wikipedia.org/wiki/Variable-length_quantity
except that this algo limits the length to 24 bits. It was used a lot
with
MIDI, at a time when hardware memory was costly (eg inside hardware
synthesizer or workstations).
What I was trying to do was pack things into 7 bits so I could recode
0's as 128. I finally thought clearly about it and realized that I only
needed to use one particular byte value (I chose 127) to duplicate so I
could repack things with a string of 0's replaced by 127 followed by the
length (up to 126) of zeros, and for 127 itself I'd just emit 127
twice. This was to pack binary data into a string that C routines
wouldn't think had ended partway through. (If I get more than 127 zeros
in a row, I just have more than one packing code.)
What is: Orphan format arguments: args[0..1]
What is: Orphan format arguments: args[0..1]
It appears to come from within unittest at the line:
strings="{0}".format(cast(int)d2[i]);
d2 is:
ubyted2[];
It should be 512 bytes long, but that hasn't been checked at the point
of the error.
The compilation used was:
rdmd --main -unittest -Dddocs blockf.d
(but dmd gave the same error on a slightly mover complex version)
The compiler was:
DMD64 D Compiler v2.066.1
Copyright (c) 1999-2014 by Digital Mars written by Walter Bright
and the system was debian testing Linux.
Re: What is: Orphan format arguments: args[0..1]
On 03/15/2015 12:27 PM, anonymous via Digitalmars-d-learn wrote:
On Sunday, 15 March 2015 at 18:46:52 UTC, Charles Hixson wrote:
What is: Orphan format arguments: args[0..1]
It appears to come from within unittest at the line:
strings="{0}".format(cast(int)d2[i]);
It means you gave `format` more arguments than placeholders. `format`
uses C style % placeholders, not the brace syntax like C#(?).
So make that:
strings="%s".format(cast(int)d2[i]);
By the way, I don't see a need for the cast.
Thanks. (Actually it was like Python...but I had remembered it as D.)
struct / cast / ? design problem
I've got, say, a file header in a routine that looks like:
structBlockHead
{ uintmagic=20150312;//block magic
uintmagic2;//magic for use by
the using routine
uintblockSize;
uintunused1;
ulongunused2;
ulongspare[61];
}
The "unused" vars are intended to be reserved for future use. The
"spare" vars are intended for use by another routine that uses the
routines of the owner of BlockHead to deal with basic things. So it
needs to define the data in spare.
How?
The only almost nice way I've thought of is with a struct pointer to
spare, and calling that nice is a statement about how bad the other ways
I've thought of are. (E.g., redefining spare as a ubyte vector and then
casting the struct of the using program as a ubyte vector and copying it
over.)
I'm sure that there must be a better way, but I can't think of what it
is. BlockHead needs to be a struct to allow the data to be written with
a writeBlock. I don't want to use a buffered file because this needs
random access. Perhaps I just shouldn't define spare, but then I
wouldn't retrieve the data with the readBlock. So I'd double the number
of disk accesses (to the BlockHead).
One alternative is to have each using routine define it's own header
struct, and to cast each BlockHead to
the appropriate routine's own header type, but that begs for errors to
creep in if any of the structs isn't the right length, of ever becomes
modified to not be the correct length.
Every way I've thought of looks like something that's just begging for
errors to creep into the code, if not now then later...so *is* there a
good way?
Re: struct / cast / ? design problem
On 03/15/2015 04:51 PM, ketmar via Digitalmars-d-learn wrote:
On Sun, 15 Mar 2015 16:34:14 -0700, Charles Hixson via Digitalmars-d-learn
wrote:
if you know the exact layouts of `spare`, you can use union for that:
struct S {
// ...
union {
ulong[61] spare;
struct { int vala; ubyte valb; }
// etc.
}
}
and then you can use it like this:
auto s = S();
s.vala = 42;
assert(s.spare[0] == 42);
you can also write a CTFE function that builds struct to cast where
`spare` is changed to something else, using `S` as a source, but this
solution will not be pretty. ;-)
The original method had no knowledge of the layout that the using module
will need, merely that it will need to store some information. (In this
particular case I *could* just modify the header in the original file,
but that isn't solving the design problem, and means that working code
needs to be modified to accommodate new code in a different file.)
IOW, if I could write the union, I wouldn't have needed to label the
unused header memory "spare". That solution *would* allow for multiple
secondary users, with different union values, but it would again mean
that new code would require the modifying of the file containing
existing code. This is normally considered bad practice.
Re: struct / cast / ? design problem
On 03/16/2015 09:16 AM, Charles Hixson via Digitalmars-d-learn wrote:
On 03/15/2015 04:51 PM, ketmar via Digitalmars-d-learn wrote:
On Sun, 15 Mar 2015 16:34:14 -0700, Charles Hixson via
Digitalmars-d-learn
wrote:
if you know the exact layouts of `spare`, you can use union for that:
struct S {
// ...
union {
ulong[61] spare;
struct { int vala; ubyte valb; }
// etc.
}
}
and then you can use it like this:
auto s = S();
s.vala = 42;
assert(s.spare[0] == 42);
you can also write a CTFE function that builds struct to cast where
`spare` is changed to something else, using `S` as a source, but this
solution will not be pretty. ;-)
The original method had no knowledge of the layout that the using
module will need, merely that it will need to store some information.
(In this particular case I *could* just modify the header in the
original file, but that isn't solving the design problem, and means
that working code needs to be modified to accommodate new code in a
different file.)
IOW, if I could write the union, I wouldn't have needed to label the
unused header memory "spare". That solution *would* allow for
multiple secondary users, with different union values, but it would
again mean that new code would require the modifying of the file
containing existing code. This is normally considered bad practice.
My current best answer is to turn the "unused" into a vector of bytes,
and then pass that to the using routines as a ref. This still is wide
open to errors at the calling end, but the BlockHead end can ensure that
the length of the vector remains constant whenever it accesses it (well,
whenever the head is being written to disk. The byte vector would be a
static array at the BlockHead end. This would seem to allow the using
end to cast the byte vector into an appropriate struct for its own use,
and that writes to it would be seen by BlockHead as writes to the
header...but only to restricted parts of the header. The problem here
is that the ref array might allow itself to be resized at the user end.
That's not a problem as long as the resizing is to something smaller,
but I'm not sure what happens if it gets resized to something larger.
It looks like allowing a buffer overflow.
Re: struct / cast / ? design problem
On 03/16/2015 11:55 AM, ketmar via Digitalmars-d-learn wrote:
On Mon, 16 Mar 2015 11:18:16 -0700, Charles Hixson via Digitalmars-d-learn
wrote:
My current best answer is to turn the "unused" into a vector of bytes,
and then pass that to the using routines as a ref. This still is wide
open to errors at the calling end, but the BlockHead end can ensure that
the length of the vector remains constant whenever it accesses it (well,
whenever the head is being written to disk. The byte vector would be a
static array at the BlockHead end. This would seem to allow the using
end to cast the byte vector into an appropriate struct for its own use,
and that writes to it would be seen by BlockHead as writes to the
header...but only to restricted parts of the header. The problem here
is that the ref array might allow itself to be resized at the user end.
That's not a problem as long as the resizing is to something smaller,
but I'm not sure what happens if it gets resized to something larger. It
looks like allowing a buffer overflow.
if you passing the array, not a slice, it can't be resized. i.e.:
void foo (ref ubyte[8] a) {
a[2] = 42;
//a.length = 6; // this will not compile: constant a.length is not an
lvalue
}
void main () {
ubyte[8] aa;
assert(aa[2] == 0);
foo(aa);
assert(aa[2] == 42);
}
Yes, but if I pass an array, the using program can't change it's own
values (which need to be stored in the header). Additionally, if I pass
an array I'm passing over 400 bytes rather than around 16, but that's
very secondary.
So, attempting to rewrite your example into more what I am talking about:
importstd.stdio;
structtstHead
{
ubytedata[400];
refubyte[400]value()
{returndata;}
voidwrite(int i)
{writefln ("%d", data[i]);}
}
voidmain()
{tstHeadtst;
autoval=tst.value;
val[23]=23;
writefln("%d", val[23]);
tst.write(23);
tst.write(0);
}
Yields:
23
0
0
instead of:
23
23
0
And:importstd.stdio;
structtstHead
{
ubytedata[400];
voidvalue(ref byte[400] val)
{val.ptr=data;}
voidwrite(int i)
{writefln ("%d", data[i]);}
}
voidmain()
{tstHeadtst;
ubyteval[400];
tst.value (val);
val[23]=23;
writefln("%d", val[23]);
tst.write(23);
tst.write(0);
}
Yields:
test.d(8): Error: val.ptr is not an lvalue
test.d(17): Error: function test.tstHead.value (ref byte[400] val) is
not callable using argument types (ubyte[400])
Failed: ["dmd", "-unittest", "-Dddocs", "-v", "-o-", "test.d", "-I."]
It is *necessary* that the using routine be able to store its data into
the parts of the header reserved for it. Perhaps multiple copying will
be necessary. I could write getter and setter routines, but they will
inevitably be doing a lot of excess copying as the base routine doesn't
know what data the calling routine will need. Every solution I've
thought of either requires a lot of excess copying, or requires a very
tight coupling between the "library" routine and the using routine.
Re: struct / cast / ? design problem
On 03/16/2015 01:24 PM, via Digitalmars-d-learn wrote:
The problem in your example is that your making a copy of the returned
data. Of course any changes to that copy won't affect the original.
You need to return a pointer to it (`ref` won't do if you want to
store it in a local variable, because these can't be `ref`).
struct BlockHead
{
uint magic = 20150312;
uint magic2;
uint blockSize;
uint unused1;
ulong unused2;
ulong spare[61];
T* embedded(T)() {
static assert(T.sizeof < spare.length);
return cast(T*) spare.ptr;
}
}
How to use it:
void useIt(ref BlockHead bh) {
static struct InternalData {
int foo;
ubyte[20] bar;
}
auto data = bh.embedded!InternalData;
data.foo = 42;
}
Nice! That looks like exactly what I was hoping for.
How to test for type without refs?
I thought that in:
T* user(T)()
{static assert (T.sizeof < bfHdr.user_.length);
static assert (__traits(isPOD, T) );
returncast(T*) bfHdr.user_.ptr;
}
the line:
static assert (__traits(isPOD, T) );
would test for there not being any embedded refs, pointers, etc., but
unittest
{
...
structthead2{stringtest;}
autouserTest2=bf.user!(thead2)();
}
executes without complaint. Is there a better way?
Re: How to test for type without refs?
On 03/17/2015 07:27 AM, anonymous via Digitalmars-d-learn wrote:
On Tuesday, 17 March 2015 at 14:06:19 UTC, Charles Hixson wrote:
I thought that in:
T* user(T)()
{static assert (T.sizeof < bfHdr.user_.length);
static assert (__traits(isPOD, T) );
returncast(T*) bfHdr.user_.ptr;
}
the line:
static assert (__traits(isPOD, T) );
would test for there not being any embedded refs, pointers, etc., but
unittest
{
...
structthead2{stringtest;}
autouserTest2=bf.user!(thead2)();
}
executes without complaint. Is there a better way?
There's std.traits.hasIndirections
.
Thanks. That did the job. (I was looking at
...dmd-doc/html/d/traits.html instead.)
buffer to struct type conversion...TArrayStream?
I've read a chunk of data into a buffer and want to convert it into a
struct. The reading routine is in a class that doesn't know about the
struct, but the size should be exactly the same. (I.e., I want to use
the converse procedure to write it.)
Is there a better way to do this than using TArrayStream?
The idea here is to have a file of fixed length records (rather like
Fortran binary records, except that there's a header record which is a
different size and which specifies various things about the file). The
class (well, struct) that handles the fixed length records only knows
what the record size is, and a couple of other quite general things.
The class which uses it holds each record in a fixed length struct with
no indirections. So I thought I could just cast the buffer to the
struct...but this doesn't work. Every straightforward way I've tried of
doing it yields:
Error: cannot cast from Node_ to ubyte[]
or something reasonably analogous. The current version of the
(non-working) code is:
ubytebuf[];
autolen=btFile.read(nodeId, buf);
assert(len == n.self.sizeof);
n.self=to!(Node.Node_)(buf);
//TODOwrite the code
which yields the error:
Error: template instance std.conv.to!(Node_).to!(ubyte[]) error
instantiating
Node_ is (approximately, I've renamed aliased values to their base value):
structNode_
{ ulongidvalue;
ulongkeyvalue;
inteLen;
Entry e[23];
}
and Entry is (approximately):
structEntry
{ ulongkey;
ulongd;
ulongd2;
}
Re: buffer to struct type conversion...TArrayStream?
On 03/19/2015 11:18 AM, ketmar via Digitalmars-d-learn wrote: On Thu, 19 Mar 2015 10:47:05 -0700, Charles Hixson via Digitalmars-d-learn wrote: turn it 90 degrees. ;-) auto cvt = cast(Node_*)buf.ptr; n = cvt[0]; Whee! Thanks, I don't think I *ever* would have thought of that. I got as far as getting a pointer to buf, but then couldn't figure out how to turn the Node_* into a Node. (Obvious now that you showed me, and I even think I've seen that trick done before.)
Re: buffer to struct type conversion...TArrayStream?
On 03/19/2015 12:05 PM, via Digitalmars-d-learn wrote:
On Thursday, 19 March 2015 at 18:42:03 UTC, Marc Schütz wrote:
3) Using std.bitmap.peek(), which also supports conversion between
big- and little-endian:
import std.bitmap;
n.self = buf.peek!(Node.Node_, Endian.bigEndian);
(The examples are untested, it's possible you'll need to make some
adjustments to make them work.)
You should probably use option 3). It is safe, because it checks that
the buffer has the right size, and it also allows you to specify the
endian-ness (many file formats have a standardized endian-ness).
While you're at it, you can also try std.bitmanip.read(). It can be
applied to any range, so you can probably do something like this
(also untested):
auto input = btFile.byChunk(4096).joiner;
while(!input.empty) {
auto node = input.read!(Node.Node_, Endian.bigEndian);
// use `node`
}
Urgh... it seems `peek()` and `read()` only work with numerical types
:-( Is this intentional? It would be quite useful for arbitrary types,
IMO, even if care must be taken with regard to pointers.
Note: Thanks for the prior comment about where to place the array
markers. I keep forgetting.
Umnf...I don't plan on needing conversion between Endian versions, but I
guess that's important. But more significant if it only handles numeric
types I wonder about handling arrays of structs, even if those structs
*are* composed entirely of numerical types. Being secure against a
future need to handle Endian variations would be good, but not enough to
both increase the complexity that way and deal with possible unexpected
errors.
P.S.: Looking at the std.bitmanip documentation causes me to feel that
the restriction to integral types was intentional, though admittedly it
doesn't seem to be stated explicitly, but all of the examples seem to be
simple arrays. I'll grant that this could be just to keep things
simple. And the wording of the docs causes me to think it would
probably only work for integral values, as in not even floats. This is
reasonable if you are thinking of it as a set of routines for bit
manipulation.
At all events, I think that it involves excessive overhead (in code
verbosity) and that I'ld likely need to use a loop to read the array
within the struct. A simple bit copy is much more straightforwards (I
was already checking that the read was the correct length, though I
haven't decided what to do if that ever fails. Currently it's an assert
statement, but this clearly needs to be changed to either an enforce
statement or to a thrown exception...but I haven't yet thought of a
valid case where the block would be an incorrect length.
Accessing a field of a containing class from within a nested class
The class Node is contained within the struct BTree. The field btFile is contained within the struct BTree. The statement is within a function within the Node class. I've tried many variations, here are a few: btFile.write(self.nodeId, cast(void*)&(self)); results in: need 'this' for 'btFile' of type 'BlockFile' this.btFile.write(self.nodeId, cast(void*)&(self)); results in: Error: no property 'btFile' for type 'btplus.BTree.Node' this.BTree.btFile.write(self.nodeId, cast(void*)&(self)); results in: Error: constructor btplus.BTree.this (string fName) is not callable using argument types (Node) Perhaps BTree needs to be a class? I made it a struct because I want it to definitely close properly when it goes out of scope.
Re: Accessing a field of a containing class from within a nested class
On 04/01/2015 11:39 AM, anonymous via Digitalmars-d-learn wrote: On Wednesday, 1 April 2015 at 18:26:49 UTC, Charles Hixson wrote: Perhaps BTree needs to be a class? yes Thanks. Sigh. I was hoping to preserve the determinate closing that one gets with a struct.
Re: Accessing a field of a containing class from within a nested class
Yess.but there are LOTS of nodes/BTree, and each node would need to
check whether the btFile was already initialized, et (ugh!) cetera. So
while possible, that's an even worse answer than depending on people
closing the BTree properly. I *am* going to separate the close routine
from the destructor, so that it can be closed manually, and if not the
destructor is going to print a warning message if the file is still open
when it's called, but this isn't the best design. This design calls for
users of what is essentially a library routine to remember to properly
close it...or to hope that the destructor gets called before the program
quits.
In a practical sense, since I'm probably going to be the only user of
the code, making BTree a class should be ok, but it feels like a bad
design decision. I don't want to make the file a static member of the
Node class (which would answer some of the problems) because that would
prevent there being more than one BTree open at a time, and I expect to
need at least two of them, and possibly more. Were I to do that I might
as well make it a file level variable.
So the ways to handle lack of a determinate close to the file are all
clumsy, and a bit error prone. But I think having a file handle in each
Node would be MUCH worse. How could you *ever* know when to close it,
as Nodes are being opened and freed all the time. And most of the time
many are open at once. Additionally, all the Nodes being closed doesn't
mean the BTree is closed (though that's the most likely reason) so you
can't even use a global counter.
OTOH, this is still in the design stage, so if a good answer were to be
available, now would be the best time to put it in. (FWIW both the Key
and the Data will be required to pass !hasIndirections!(T). I'm not
planning a truly general BTree. In my test version the Key is a ulong
and the Data is a struct containing only ulongs and ints.)
On 04/01/2015 03:03 PM, Ali Çehreli via Digitalmars-d-learn wrote:
On 04/01/2015 11:25 AM, Charles Hixson via Digitalmars-d-learn wrote:>
The class Node is contained within the struct BTree.
> The field btFile is contained within the struct BTree.
> The statement is within a function within the Node class.
>
> I've tried many variations, here are a few:
>
> btFile.write(self.nodeId, cast(void*)&(self));
> results in:
> need 'this' for 'btFile' of type 'BlockFile'
>
> this.btFile.write(self.nodeId, cast(void*)&(self));
> results in:
> Error: no property 'btFile' for type 'btplus.BTree.Node'
>
> this.BTree.btFile.write(self.nodeId, cast(void*)&(self));
> results in:
> Error: constructor btplus.BTree.this (string fName) is not callable
> using argument types (Node)
>
> Perhaps BTree needs to be a class? I made it a struct because I
want it
> to definitely close properly when it
> goes out of scope.
Can you modify the definition of Node? If so, perhaps it's possible
construct Node objects with a reference to its btFile:
import std.stdio;
class Node
{
int *btFile;
this(int *btFile)
{
this.btFile = btFile;
}
void foo()
{
writeln(*btFile);
}
}
struct BTree
{
int btFile;
Node node;
this(int btFile)
{
this.btFile = btFile;
this.node = new Node(&this.btFile);
}
}
void main()
{
auto bt = BTree(42);
bt.node.foo();
}
Ali
Re: Efficiently passing structs
On 05/05/2015 11:49 AM, Ali Çehreli via Digitalmars-d-learn wrote: On 05/05/2015 07:14 AM, bitwise wrote: > I don't see how someone could arrive at the above > solution without showing up here and asking first. It was the same with me. :) Then I wrote a short section about it: http://ddili.org/ders/d.en/lvalue_rvalue.html#ix_lvalue_rvalue.auto%20ref,%20parameter Ali Are there plans to make this as a book rather than an e-book? I find books much more useful.
How to simulate a new type
What are the downsides to simulating a new type with a struct.
What I have in mind is something along the lines of:
struct myType
{ uint64_t value;
}
The goal of this type is to prevent accidental conversions from myType
into ints, uint64_ts, etc.
Re: How to simulate a new type
On 05/14/2015 01:42 PM, Laeeth Isharc via Digitalmars-d-learn wrote:
On Thursday, 14 May 2015 at 18:42:56 UTC, Charles Hixson wrote:
What are the downsides to simulating a new type with a struct.
What I have in mind is something along the lines of:
struct myType
{ uint64_t value;
}
The goal of this type is to prevent accidental conversions from
myType into ints, uint64_ts, etc.
http://dlang.org/phobos/std_typecons.html#.Typedef
Yes, that looks as if it would do the job, but what are its advantages
over a simple struct? I'd be hesitant to try to predict the RAM layout
of a templated variable, but it's my understanding that a struct has
neither compile time nor runtime penalties over a simple variable, and
that it doesn't increase RAM comsumption when included within another
struct, or in a static array.
What I was really wondering was whether the limitation of access caused
by enclosing a variable within a struct was worth the benefit.
Templates add a whole 'nother layer of complexity. For that matter I
was thinking that, if I so chose, I could at some point add limits along
the lines of Ada's types. I like being *able* to define conversions for
types, but frequently I'd rather not have them arbitrarily assumed.
Otherwise I'd just use alias, which almost does what I want. (E.g., one
thing I might eventually want to do is use a couple of bits at the top
of the uint64_t as flags, so if I did define operations I'd need to be
able to mask the flag bits off before the operation and reapply them to
the result, in some defined way of combining the flags. Not good if
sometimes the value is going to be automatically reinterpreted as a
plain number.)
Re: How to simulate a new type
On 05/14/2015 06:38 PM, Adam D. Ruppe via Digitalmars-d-learn wrote: On Friday, 15 May 2015 at 01:03:32 UTC, Charles Hixson wrote: Yes, that looks as if it would do the job, but what are its advantages over a simple struct? None really, except perhaps automatic forwarding of operators which is easy enough to do on a struct too (and with a struct, you can do only the ones that actually make sense for you). Typedef has a few major disadvantages too - surprising behavior if you forget the second parameter, for example. The library typedef should really be removed. Thank you. I was hoping that struct was a good approach, I've just never seen anyone recommending it, so I wanted to check.
Re: Ada-Style Sub-Typing
It's not totally convenient, but what I'm currently using is a slightly
less elaborate struct:
structA
{intv;}
Since all I'm creating is an isolated type, the private stuff isn't
needed. If I wanted to get fancy I could start implementing
operations. Comparison is free, as sturcts of the same type are, by
default, compared via a bit match. This, however, doesn't give you less
than, greater than, etc. I think you need to implement those, almost
certainly if you want to use anything but unsigned comparison.
OTOH, if you do an alias this, then I'm not sure you maintain type
isolation. I think that will auto-convert to the type of the variable.
(I haven't checked this...so perhaps I'm just being pessimistic/optimistic.)
Consider what purpose declaring the internal variable to be private
serves. Remember that structs are copied rather than passed by
reference, and I think you'll decide it doesn't really serve any purpose.
On 06/02/2015 01:16 PM, via Digitalmars-d-learn wrote:
Is there some elegant way of creating "isolated" types in D similar to
the semantics of `subtype` in Ada.
Something like this
struct A
{
this(int value) { this._value = value; }
private int _value;
alias _value this;
}
struct B
{
this(int value) { this._value = value; }
private int _value;
alias _value this;
}
void main(string[] args)
{
A a = 11;
B b = 12;
a = b;
}
except that
the final statement should error.
std.concurrent Tid vector initialization problem
I'm planning an application where a series of threads each need to be aware of the Tids of all the others. The number won't be known at compile time, but that doesn't seem to change the design. All I've been able to come up with is a pair of loops, one to spawn the threads, and collect their Tids, and a second for each one to send its Tid to all the others. receive doesn't seem to want to work with shared Tids. My original design was to have a file level shared array of Tids, but receive (or possibly send) objected to attempts to use them. They aren't known until they've been spawned, so I can't pass them as spawn parameters. Etc. This seems like a very clumsy initialization design. Does anyone have a better idea? (The rough estimate of the number of Tids is six, but that's likely to change from run to run.)
std.concurrent Tid vector initialization problem
I'm planning an application where a series of threads each need to be aware of the Tids of all the others. The number won't be known at compile time, but that doesn't seem to change the design. All I've been able to come up with is a pair of loops, one to spawn the threads, and collect their Tids, and a second for each one to send its Tid to all the others. receive doesn't seem to want to work with shared Tids. My original design was to have a file level shared array of Tids, but receive (or possibly send) objected to attempts to use them. They aren't known until they've been spawned, so I can't pass them as spawn parameters. Etc. This seems like a very clumsy initialization design. Does anyone have a better idea? (The rough estimate of the number of Tids is six, but that's likely to change from run to run.)
Re: std.concurrent Tid vector initialization problem
On Sunday, 28 June 2015 at 09:59:29 UTC, Marc Schütz wrote:
On Sunday, 28 June 2015 at 01:02:02 UTC, Charles Hixson wrote:
I'm planning an application where a series of threads each
...gn. Does anyone have a better idea?
(The rough estimate of the number of Tids is six, but that's
likely to change from run to run.)
The most elegant solution I can think of is this:
struct Go { }
__gshared const Tid[] allTids;
void myfunc(...) {
receiveOnly!Go();
// do the work
}
shared static this() {
Tid[] tids;
foreach(thread; threads)
tids ~= spawn(&myfunc, ...);
*cast(const(int)[]*) &allTids = tids;
foreach(tid; allTids)
tid.send(Go());
}
I believe the cast is not even undefined behaviour, because
it's not immutable, but I'm not sure. But the const-ness guards
against accidental modification of `allTids` by a thread.
Alternatively, you could make `allTids` private, provide a
public getter, and implement the threads in another module.
Or you could simply leave the global array mutable and be
careful not to modify it.
Thanks. That sounds similar to the approach that I had though
of, except that you're doing it in a static this().
(I'd have answered sooner, but my D email seems to have died.
Only the D groups are currently dead now, though my entire
account was dead for a couple of weeks. [AT&T])
What is the std.stream replacement?
I have DMD64 D Compiler v2.067.1 installed, and in the documentation of phobos what it says about std.stream is "don't use it on new code". It doesn't, however, appear to offer any replacement. Certainly std.file, std.stdio, and std.path aren't replacements. So what *is* the appropriate replacement? More specificly, what should one use for binary i/o, particularly random access binary i/o? With fixed block size.
Re: Replacement of std.stream
On Monday, 29 June 2015 at 12:00:14 UTC, Jonathan M Davis wrote: On Sunday, June 28, 2015 11:14:57 Baz via Digitalmars-d-learn wrote: On Sunday, 28 June 2015 at 05:04:48 UTC, DlangLearner wrote: > I will convert a Java program into D. The original Java code > is based on the class RandomeAccessFile which essentially > defines a set of methods for read/write > Int/Long/Float/String etc. The module std.stream seems to be > a good fit for this job, but in its documentation, it is > marked deprecated. So I'd like to know what is the > replacement for this module. If I don't use this module, > what is the other apporach I should use. Thanks for help. You can use std.stream. There is no candidate to replace it. Even if tomorrow someone comes with one, it has to be reviewed, accepted in std.experimental and after a while it would totally replace the old one. I think it's safe to say that std.stream will exist for at least 2 years in its current shape. No. It was decided at dconf that it was going to be deprecated and then removed within a relatively short period of time. In fact, it would have been deprecated with the next release, but there was a problem with the deprecation and the ddoc build, so the deprecation was temporarily reverted. But once that's sorted out, it's going to be deprecated, and it won't be around much longer at all. We decided that it was worse to leave it in saying that it was going to be replaced without having come up with anything for years than to leave it in. And given that std.stream rarely even comes up in discussions and that no one has proposed a replacement, Andrei thinks that it's a sign that there really isn't much demand for it anyway. For the most part, simply using std.stdio.File or std.mmfile.MmFile with ranges does what streams need to do just fine. Maybe someone will come up with a replacement for std.stream eventually, but there really doesn't seem to be much call for it. Regardless, until someone comes up with a replacement, we're simply not going have a stream-based I/O (though ranges are close - which is part of why no one has felt the need to replace std.stream strongly enough to actually do it). - Jonathan M Davis The documentation of std.mmfile.MmFile makes it an unacceptable replacement for std.stream. I can't even tell if it's features are missing, or just unintelligible. It rather looks as if you need to have enough RAM to hold the entire file in memory at once, but I'm not even sure of that. Perhaps this is just a documentation problem. Perhaps. But it's certainly a real problem. std.stdio.File is a much more reasonable substitute, but the documentation needs to be improved...probably lifted into a major heading, perhaps std.stdFile or some such. And examples. As it is it's buried where it gets overlooked...of course, that would tend to break any code that depended on using it is where it is now, so this should be done quickly, or this will end up being a continuing problem. (Even as it is, much of my code will stop compiling when this change is finalized, as I have used std.stream most of the time for years, not knowing of any alternative.)
std.uni general character category
In std.uni (D Lib 2.068.2) I can no longer see how to get the general category code for a character. Does anyone know what the currently supported way to do that is?
Re: std.uni general character category
On 10/20/2015 10:38 AM, Charles Hixson via Digitalmars-d-learn wrote:
In std.uni (D Lib 2.068.2) I can no longer see how to get the general
category code for a character. Does anyone know what the currently
supported way to do that is?
I thought I remembered that I used to be able to directly get the
general character category, but as a crutch this is what I'm using (and
checking back to 2012 I apparently couldn't do better then):
charcharCat (dchar ch)
{ if(isAlpha (ch) )return'L';
if(isNumber (ch) )return'N';
if(isWhite (ch) )return'Z';
if(isControl (ch) ) return'C';
if(isPunctuation (ch) ) return'P';
else return'?';// Includes
not a character
}
This suffices for my current needs, but it clearly a lot less
information than the two letter code would be, and sometimes that's what
I need.
error detected at """ ch in unicode.C """ Library error?
To me this looks like a library error, but I'm not sure. Any suggestions
importstd.uni;
chargcCat1(dchar ch)
{ if(ch in unicode.L)return'L';//Letter
if(ch in unicode.M)return'M';//Mask
if(ch in unicode.C)return'C';// Control
<<== error here!
if(ch in unicode.N)return'N';// Numeric
if(ch in unicode.P)return'P';// Punctuation
if(ch in unicode.S)return'S';// Symbol
if(ch in unicode.Z)return'Z';// Separator
return'?';
}
$ rdmd --main -unittest test2.d
/usr/include/dmd/phobos/std/uni.d(6220): Error: slice [0..2] exceeds
array bounds [0..1]
/usr/include/dmd/phobos/std/uni.d(6220):called from here:
comparePropertyName(name[0..2], "In")
/usr/include/dmd/phobos/std/uni.d(6119):called from here:
findAny("C")
/usr/include/dmd/phobos/std/uni.d(6122): Error: static assert "No
unicode set by name C was found."
test2.d(7):instantiated from here: opDispatch!"C"
Failed: ["dmd", "-unittest", "-v", "-o-", "test2.d", "-I."]
$ dmd
DMD64 D Compiler v2.068.2
Copyright (c) 1999-2015 by Digital Mars written by Walter Bright
Documentation: http://dlang.org/
Config file: /etc/dmd.conf
...
