Re: Dare I ... another volatile discussion ?
On Saturday, 9 May 2015 at 16:59:35 UTC, Jens Bauer wrote: ... System calls will need to access the peripherals in some way, in order to send data to for instance a printer or harddisk. If the way it's done is using a memory location, then it's necessary to tell the compiler that this is not ordinary memory, but I/O-memory AKA hardware address space. Userland code still uses system calls and not global variables, whatever is expressed in read(2) signature tells the compiler enough to pass data via buffer. Shared is supposed to prevent the programmer from accidentally putting unshared data in a shared context. Expectedly people wanted it to be a silver bullet for concurrency, instead std.concurrency provides high-level concurrency safety. In other words, it's the oposite of 'static' ? Whether data is shared or not is not tied to its storage class, that's why its shared nature is expressed in its type and storage class can be anything; for the same reason shared type qualifier is transitive.
Re: Dare I ... another volatile discussion ?
On Sunday, 10 May 2015 at 12:43:31 UTC, Kagamin wrote: On Saturday, 9 May 2015 at 16:59:35 UTC, Jens Bauer wrote: ... System calls will need to access the peripherals in some way, in order to send data to for instance a printer or harddisk. If the way it's done is using a memory location, then it's necessary to tell the compiler that this is not ordinary memory, but I/O-memory AKA hardware address space. Userland code still uses system calls and not global variables, I think it is essential to emphasize I/O-space is not, and can not be compared to variables. Variables reside in RAM. I/O-space is outside RAM and usually not accessible for anything but kernel and drivers. On simple microcontrollers, there's no user and supervisor modes; thus I/O-space can be accessed from any part of the program. You could say that such microcontrollers are always running in 'supervisor mode' or 'privileged mode'. whatever is expressed in read(2) signature tells the compiler enough to pass data via buffer. Yes. If we take 'read' as an example, the system call takes your data-block and at some point it transfers your data block to the 'driver'. The driver receives the data, but where does it put the data, in order to write the data to your harddisk ? On some systems, it writes to a harddisk controller, which resides in I/O-space. This harddisk controller is not software, it's hardware. It means the data written to I/O-space, AKA. Hardware registers, go directly out onto the PCB traces and head for an external chip outside the CPU; the chip is a bridge chip, which is only the middle-man. The data is passed on to another chip connected to the bridge, and this other chip will then see oh, it's a command, that I should move the arm that holds the harddisk's read/write head; it also receives a position. This command may be 2 bytes in size. A series of such commands are necessary before the actual data can begin to be transferred. When the harddisk head is in the right position and all the other preparations have been made, the CPU can start transferring the data, which is held in the buffer in RAM. It may transfer the data byte-by-byte, until all data have been transferred. I'm sorry for such a tedious old-fashioned example, but it really explains it the best way. Today, we have DMA; we set a pointer and a length, and give the command Begin, and the data is transferred automatically, so the CPU is actually free to do other things while our transfer is being done in the background - but the basics are the same. To tell the DMA where to start transferring from and how many bytes to transfer and trigger the transfer, one has to write to I/O-space (on most systems). Variables are not at all behaving like peripherals, because they reside in normal RAM (well, usually they reside in RAM, especially on computers). Imagine you write value 0xA5 to address 0x3840. If you have all interrupts disabled and read the value immediately after that, what do you expect to read ? Yes, of course, you expect to read 0xA5. But you will never read that value, because the hardware always reads this particular I/O location as 0x13. Now, your written value, somehow is readable in address 0x382C. So whatever you write in 0x3840 is immediately readable in address 0x382C. If this is confusing, then wait until you hear this: Some peripherals require you to *read* an address in order to clear an interrupt-pending bit. As soon as you read it, the pending bit will be cleared and you can have another interrupt, but not before that happens. This might sound wierd, but how about this: In another location on the same chip, you'll find that you need to write a one, in order to clear a bit in a register. However, not all bits act that way in this 32-bit register; some are 'sticky', some bits are read-only and some are write-one-to-clear. Alright, alright, we're crazy enough now, aren't we ? -NO! :) Another kind of peripheral resets a counter, whenever you write *any* value to a hardware register. It does not matter which value; pick one, then the address you write to will be cleared to 0. Reading it immediately after will either give you the result 0 or perhaps 1 if it already started counting. ... That means I/O-space isn't about atomic operations nor is it only about writing the values in the right order, but it may be completely nuts and twist your brain. :) Thus ... Having this kind of space, which is addressable the same way RAM is addressable, but behaves dramatically different, one needs to be able to tell the compiler that this is not RAM and cannot behave as RAM. C does this by using the 'volatile' keyword, which is often just used to share variables between task-time and interrupt-time. But what volatile is really about is to tell the compiler that ... 1: It's not allowed to do *any* caching of the value 2: It can not predict which value the address may
Re: Dare I ... another volatile discussion ?
On Thursday, 7 May 2015 at 21:42:08 UTC, Iain Buclaw wrote: On 7 May 2015 at 23:39, Iain Buclaw ibuc...@gdcproject.org wrote: When used properly though, it's properties make it a prime candidate for the foundation of libraries/programs that centre around the use of atomics. However, shared is not to be confused with a thread-safe atomic type. Thread-safety is left to the end-user on such matters of access. That last sentence should also include a mention of memory ordering too being left to the end-user. Thank you for explaining this; it does make sense. :) -So 'shared' is not to be confused with 'atomic'; its behaviour seems closer to C's extern, do I understand this correctly ? One of the reasons I asked the question, is because no doubt D will need to write to hardware on any kind of platform, and I'd like it to be the same on both microcontrollers and a Personal Computer (not limited to a specific kind of processor). Another thing, slightly related, is 'atomic'. C's implementation allows you to have a function, which reads/modifies/writes a value atomically. This is all implemented 'outside' the C language. Would it make sense to mark a variable 'atomic', or would it be rather crazy, because if it's extern(C) anyway, C wouldn't access it atomically ?
Re: Dare I ... another volatile discussion ?
On Thursday, 7 May 2015 at 16:04:56 UTC, Jens Bauer wrote: Regarding (1), because marking a variable 'shared' is not enough (it allows instructions to be moved around), Johannes already made a volatileLoad and volatileStore, which will be usable for microcontrollers, though for convenience, it requires writing additional code. -But this solution ... I do not know if it would work, when writing a driver for Debian running on a i586 platform or PowerMac G3 for instance. System calls on sufficiently smart processors like x86 use C-like ABI good practices: registers and buffers to pass data instead of global variables, because multithreaded programs will have race condition on accessing the global variables. See read(2) syscall as an example of such API http://man7.org/linux/man-pages/man2/read.2.html On Thursday, 7 May 2015 at 18:18:02 UTC, Johannes Pfau wrote: Not sure about shared (I don't think anybody knows what exactly shared is supposed to do) Shared is supposed to prevent the programmer from accidentally putting unshared data in a shared context. Expectedly people wanted it to be a silver bullet for concurrency, instead std.concurrency provides high-level concurrency safety.
Re: Dare I ... another volatile discussion ?
On Saturday, 9 May 2015 at 12:16:58 UTC, Kagamin wrote: On Thursday, 7 May 2015 at 16:04:56 UTC, Jens Bauer wrote: Regarding (1), because marking a variable 'shared' is not enough (it allows instructions to be moved around), Johannes already made a volatileLoad and volatileStore, which will be usable for microcontrollers, though for convenience, it requires writing additional code. -But this solution ... I do not know if it would work, when writing a driver for Debian running on a i586 platform or PowerMac G3 for instance. System calls on sufficiently smart processors like x86 use C-like ABI good practices: registers and buffers to pass data instead of global variables, because multithreaded programs will have race condition on accessing the global variables. See read(2) syscall as an example of such API http://man7.org/linux/man-pages/man2/read.2.html To make my question a little clearer: This part is not about RAM locations, but I/O-memory locations AKA. hardware addresses. On some systems, such as the 68xxx based Atari and Amiga, peripherals are accessed by reading and writing to memory locations; those locations are addresses belonging to hardware; eg. hardware space. This depends on the CPU. As an example, the Atari, the address space was usually 0x..ff, where the first two dots were don't care, as the 68000 was only 24-bit; later, 0x00fx was mirrored to 0xfffx, for backwards compatibility. Thus the address space between 0x..f0 and 0x..ff was not ordinary RAM, but I/O-space. On Z80, for instance, it's common to use the IN and OUT instructions to access hardware, so normally you wouldn't use precious memory locations for peripherals on such systems. ... System calls will need to access the peripherals in some way, in order to send data to for instance a printer or harddisk. If the way it's done is using a memory location, then it's necessary to tell the compiler that this is not ordinary memory, but I/O-memory AKA hardware address space. On Thursday, 7 May 2015 at 18:18:02 UTC, Johannes Pfau wrote: Not sure about shared (I don't think anybody knows what exactly shared is supposed to do) Shared is supposed to prevent the programmer from accidentally putting unshared data in a shared context. Expectedly people wanted it to be a silver bullet for concurrency, instead std.concurrency provides high-level concurrency safety. In other words, it's the oposite of 'static' ? -If so, then that makes the purpose much clearer to me, and it absolutely makes sense. :) ... Like the export symbol or xdef symbol directives in assembly language.
Re: Dare I ... another volatile discussion ?
On Thursday, 7 May 2015 at 16:04:56 UTC, Jens Bauer wrote:
So back to what originally triggered this post; this is what
the question is actually about:
If writing a driver for platform X, how is reading/writing
hardware registers usually done in D ?
Here's my pre 2.067 code for `volatile` semantics
@inline T volatileLoad(T)(T* a)
{
asm { ::: memory; };
return *cast(shared T*)a;
}
@inline void volatileStore(T)(T* a, in T v)
{
asm { ::: memory; };
*cast(shared T*)a = v;
}
As Iain showed, shared does not provide any order guarantees, so
that's why I added the asm { ::: memory; }; memory barrier.
This code, however, is still incorrect because it uses a bug in
GDC as a feature. Iain explained that bug here:
https://youtu.be/o5m0m_ZG9e8?t=3141
There's also a post here that shows a volatileLoad/Store
implementation using memory barriers without shared
(http://forum.dlang.org/post/501a6e01.7010...@gmail.com), but
when I tried it with my cross-compiler, I got an ICE.
Furthermore, I don't even understand it. I don't understand
exactly what the +g and +m constraints do, and I hate using code
I don't understand.
Once 2.067 is properly merged and implemented, none of this will
be necessary anyway as volatileLoad/Store intrinsics were
introduced. That being said, you will still have to add some
boilerplate around volatileLoad/Store to make the syntax
tolerable. Johannes has a nice implementation of Volatile!T for
that purpose here (http://dpaste.dzfl.pl/dd7fa4c3d42b). I have
my own CTFE Template implementation here
(https://github.com/JinShil/stm32f42_discovery_demo/blob/master/source/stm32f42/mmio.d).
In terms of shared, I think the following thread is quite
telling:
http://forum.dlang.org/post/lruc3n$at1$1...@digitalmars.com. Based
on that, I wouldn't trust shared for anything, and I think it
should just be ignored altogether. See also
http://p0nce.github.io/d-idioms/#The-truth-about-shared
I'm not sure what to do yet for synchronized/atomic access. If I
had to do in now, I'd probably just stick with C-like techniques.
Mike
Re: Dare I ... another volatile discussion ?
On 7 May 2015 at 18:04, Jens Bauer via Digitalmars-d
digitalmars-d@puremagic.com wrote:
I'm sorry for opening such a topic; I've heard it's not liked a lot, but I
think it might be necessary.
I'm not asking for a 'volatile' keyword, but rather to find out what the
right thing to use is.
After reading a few different threads related to microcontrollers, I started
wondering how to program the following in D:
1: System level drivers, which writes directly to hardware registers (any
architecture, PC/i386 [Linux, Windows, others], Atari ST, IBM BladeCenter,
etc.)
2: Interrupts that needs to share variables.
1) is what we basically need on microcontrollers. If it's possible to write
a driver in D, which has no problems with accessing hardware, then it should
be possible to do it for any microcontroller as well.
2) shared variables could be used for interrupts, but what happens if the
TLS is disabled; will shared variables work ? -Interrupts are not threads.
Regarding (1), because marking a variable 'shared' is not enough (it allows
instructions to be moved around), Johannes already made a volatileLoad and
volatileStore, which will be usable for microcontrollers, though for
convenience, it requires writing additional code.
-But this solution ... I do not know if it would work, when writing a driver
for Debian running on a i586 platform or PowerMac G3 for instance.
If variable 'alice' and variable 'bob' are both shared, and reading from
'bob', then writing to 'alice'; would instructions be moved around, so
reading from 'bob' could actually occur after writing to 'alice' ?
Yes. Take this example:
---
shared int alice, bob;
void foo()
{
alice = bob + 1;
bob = 0;
}
---
gdc without optimisations produces:
---
; load bob into %eax
movl_D7reorder3bobOi(%rip), %eax
; + 1
addl$1, %eax
; store to alice
movl%eax, _D7reorder5aliceOi(%rip)
; bob = 0
movl$0, _D7reorder3bobOi(%rip)
---
gdc with optimisations produces:
---
; load bob into eax
movl_D7reorder3bobOi(%rip), %eax
; bob = 0
movl$0, _D7reorder3bobOi(%rip)
; + 1
addl$1, %eax
; store to alice
movl%eax, _D7reorder5aliceOi(%rip)
---
Now, this does not change the behaviour of the program if we consider
that bob and alice do not have any side effects. However on a
micro-controller dealing with direct memory I/O ...
Iain.
Re: Dare I ... another volatile discussion ?
Am Thu, 07 May 2015 16:04:55 + schrieb Jens Bauer doc...@who.no: I'm sorry for opening such a topic; I've heard it's not liked a lot, but I think it might be necessary. I'm not asking for a 'volatile' keyword, but rather to find out what the right thing to use is. After reading a few different threads related to microcontrollers, I started wondering how to program the following in D: 1: System level drivers, which writes directly to hardware registers (any architecture, PC/i386 [Linux, Windows, others], Atari ST, IBM BladeCenter, etc.) 2: Interrupts that needs to share variables. 1) is what we basically need on microcontrollers. If it's possible to write a driver in D, which has no problems with accessing hardware, then it should be possible to do it for any microcontroller as well. 2) shared variables could be used for interrupts, but what happens if the TLS is disabled; will shared variables work ? -Interrupts are not threads. Regarding (1), because marking a variable 'shared' is not enough (it allows instructions to be moved around), Johannes already made a volatileLoad and volatileStore, which will be usable for microcontrollers, though for convenience, it requires writing additional code. 1) Actually these were proposed for DMD some time before I've implemented the GDC part ;-) Shared should be avoided for volatile data for now. It might make sense if you want to synchronize accesses from threads to a volatile memory location, but then things might get complicated. 2) This can be done with volatileLoad/Store as well. Just access a global variable using volatileLoad/Store. This can also be used nicely in a wrapper. If you want to enforce atomic access to volatile variables (bigger than the word size) you might need shared(Volatile!T). Again things might get complicated ;-) If variable 'alice' and variable 'bob' are both shared, and reading from 'bob', then writing to 'alice'; would instructions be moved around, so reading from 'bob' could actually occur after writing to 'alice' ? Not sure about shared (I don't think anybody knows what exactly shared is supposed to do) but if you use volatileLoad/Store these instructions won't be moved around.
Re: Dare I ... another volatile discussion ?
On 7 May 2015 at 20:18, Johannes Pfau via Digitalmars-d digitalmars-d@puremagic.com wrote: If variable 'alice' and variable 'bob' are both shared, and reading from 'bob', then writing to 'alice'; would instructions be moved around, so reading from 'bob' could actually occur after writing to 'alice' ? Not sure about shared (I don't think anybody knows what exactly shared is supposed to do) but if you use volatileLoad/Store these instructions won't be moved around. I've change my mind over the years, so who knows whether or not I'll change again, but this is the current definition I give to it. Shared data on an ABI level is exactly the same as __gshared, but that's where the similarities end. On a semantic level, shared comes with a transitive, and statically type-checked guarantee that either taking it's address or passing it's reference around stays within the bounds of it's given qualifier, so everything it's address passes through must also be shared. To make it an even stronger type, non-shared data can not be implicitly promoted to shared. Of course, explicit casts allow you to demote and promote the 'shared' qualifier as much as you like, circumventing all guarantee. When used properly though, it's properties make it a prime candidate for the foundation of libraries/programs that centre around the use of atomics. However, shared is not to be confused with a thread-safe atomic type. Thread-safety is left to the end-user on such matters of access. Regards Iain.
Re: Dare I ... another volatile discussion ?
On 7 May 2015 at 23:39, Iain Buclaw ibuc...@gdcproject.org wrote: When used properly though, it's properties make it a prime candidate for the foundation of libraries/programs that centre around the use of atomics. However, shared is not to be confused with a thread-safe atomic type. Thread-safety is left to the end-user on such matters of access. That last sentence should also include a mention of memory ordering too being left to the end-user.
Dare I ... another volatile discussion ?
I'm sorry for opening such a topic; I've heard it's not liked a
lot, but I think it might be necessary.
I'm not asking for a 'volatile' keyword, but rather to find out
what the right thing to use is.
After reading a few different threads related to
microcontrollers, I started wondering how to program the
following in D:
1: System level drivers, which writes directly to hardware
registers (any architecture, PC/i386 [Linux, Windows, others],
Atari ST, IBM BladeCenter, etc.)
2: Interrupts that needs to share variables.
1) is what we basically need on microcontrollers. If it's
possible to write a driver in D, which has no problems with
accessing hardware, then it should be possible to do it for any
microcontroller as well.
2) shared variables could be used for interrupts, but what
happens if the TLS is disabled; will shared variables work ?
-Interrupts are not threads.
Regarding (1), because marking a variable 'shared' is not enough
(it allows instructions to be moved around), Johannes already
made a volatileLoad and volatileStore, which will be usable for
microcontrollers, though for convenience, it requires writing
additional code.
-But this solution ... I do not know if it would work, when
writing a driver for Debian running on a i586 platform or
PowerMac G3 for instance.
If variable 'alice' and variable 'bob' are both shared, and
reading from 'bob', then writing to 'alice'; would instructions
be moved around, so reading from 'bob' could actually occur after
writing to 'alice' ?
Imagine that in the microcontroller world, there's a *bunch* of
hardware registers.
The best thing a microcontroller knows, is to read/write hardware
registers; often only a few bits at a time - but definitely also
8-, 16- and 32-bit values.
Thus you will most likely not be able to find a single piece of
source-code for a microcontroller, where it does not access
hardware. This means: Hardware is used often, and reading/writing
hardware registers should not take up a lot of space in the
source code.
For those, who are unfamiliar with such code, here's a silly
example in C on how it might look.
The following code changes the clock-frequency of the
microcontroller, so it runs at 100 MHz:
---8-8-8-
uint32_t setupCoreClock()
{
/* By default, we run at 100MHz, however,
* our PLL clock frequency is 300 MHz.
* We'd like to run 400MHz! */
LPC_SC-SCS = SCS_MainOscillatorEnable; /* enable Main
Oscillator */
if(LPC_SC-SCS SCS_MainOscillatorEnable)
{
while(0 == (LPC_SC-SCS SCS_MainOscillatorStatus)){}
}
LPC_SC-CCLKCFG = CCLK_Divider - 1;
LPC_SC-PCLKSEL0 = 0;
LPC_SC-PCLKSEL1 = 0;
LPC_SC-CLKSRCSEL = PLL0_ClockSource;
LPC_SC-PLL0CFG = PLL0_Configuration;
LPC_SC-PLL0FEED = 0xaa;
LPC_SC-PLL0FEED = 0x55;
LPC_SC-PLL0CON = PLL_Enable;
LPC_SC-PLL0FEED = 0xaa;
LPC_SC-PLL0FEED = 0x55;
while(!(LPC_SC-PLL0STAT PLL0_Lock)){} /* Wait for PLOCK0 */
LPC_SC-PLL0CON = PLL_Enable | PLL_Connect;
LPC_SC-PLL0FEED = 0xaa;
LPC_SC-PLL0FEED = 0x55;
while((LPC_SC-PLL0STAT (PLL0_EnabledFlag |
PLL0_ConnectedFlag)) != (PLL0_EnabledFlag |
PLL0_ConnectedFlag)){} /* Wait until connected */
return(F_CCLK);
}
---8-8-8-
Everything you see above, which starts with 'LPC_' are hardware
register accesses.
Not counting the curly braces, nearly every line of the code
access hardware registers.
Things like LPC_SC are pointers to structures, which contains a
load of volatile keywords; some prohibit reading, some prohibit
writing, some allow both, some allow neither.
Most code for ARM Cortex-M looks like the above, because the
silicon vendors just modifies the stock (template) source code,
which they receive from ARM. Thus there's some convenience, but
not a lot. Bitfields are not provided by ARM, so it propagates
down to the vendor, which do not supply bitfields either.
However, byte/halfword/word access (8/16/32-bit) are sometimes
provided through anonymous unions, which is very helpful.
So back to what originally triggered this post; this is what the
question is actually about:
If writing a driver for platform X, how is reading/writing
hardware registers usually done in D ?
