Re: [fpc-devel] volatile variables
On 06/29/2011 05:47 PM, Jonas Maebe wrote: ... Thanks for the multiple pointers. I was just trying to construct an example that (a) is similar to stuff an normal user might think would be sure to work and (b) if the cache-sync problems really exist in the way discussed is likely to fail on an SMP-machine so that it makes sense to do a test. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
Vinzent Höfler schrieb: Question is, what makes one variable use read/write-through, while other variables can be read from the cache, with lazy-write? Synchronisation. Memory barriers. That's what they are for. And this doesn't happen out of thin air. How else? Ok, maybe I misunderstood your question. Normally, these days every memory access is cached (and that's of course independent from compiler optimizations). This should be normal (speedy) behaviour. But there are usually possibilities to define certain areas as write-through (like the video frame buffer). But that's more a chip-set thing than it has anything to do with concurrent programming. I also thought about cache or page attributes (CONST vs. DATA sections), but these IMO don't offer the fine granularity, that would allow to separate synchronized from unsynchronized variables (class/record members!) in code. Even the memory manager would have to deal with two classes of un/synced memory then :-( Apart from that you have to use the appropiate CPU instructions. Seems to be the only solution. Is this a compiler requirement, which has to enforce read/write-through for all volatile variables? No. volatile (at least in C) does not mean that. Can you provide a positive answer instead? Ada2005 RM: |C.6(16): For a volatile object all reads and updates of the object as | a whole are performed directly to memory. |C.6(20): The external effect [...] is defined to include each read and | update of a volatile or atomic object. The implementation shall | not generate any memory reads or updates of atomic or volatile | objects other than those specified by the program. That's Ada's definition of volatile. C's definition is less stronger, but should basically have the same effect. Is that positive enough for you? ;) Much better ;-) But what would this mean to FPC code in general (do we *need* such attributes?), and what will be their speed impact? This obviously depends on the effects of the specific synchronizing instructions, inserted by the compiler. But if so, which variables (class fields...) can ever be treated as non-volatile, when they can be used from threads other than the main thread? Without explicit synchronisation? Actually, none. Do you understand the implication of your answer? I hope so. :) When it's up to every coder, to insert explicit synchronization whenever required, how to determine the places where explicit code is required? By careful analysis. Although there may exist tools which detect potentially un-synchronised accesses to shared variables, there will be no tool that inserts synchronisation code automatically for you. I wouldn't like such tools, except the compiler itself :-( Consider the shareable bi-linked list, where insertion requires code like this: list.Lock; //prevent concurrent access ... //determine affected list elements new.prev := prev; //prev must be guaranteed to be valid new.next := next; prev.next := new; next.prev := new; list.Unlock; What can we expect from the Lock method/instruction - what kind of synchronizaton (memory barrier) can, will or should it provide? My understanding of a *full* cache synchronization would slow down not only the current core and cache, but also all other caches? If so, would it help to enclose above instructions in e.g. Synchronized begin update the links... end; so that the compiler can make all memory references (at least reads) occur read/write-through, inside such a code block? Eventually a global cache sync can be inserted on exit from such a block. After these considerations I'd understand that using Interlocked instructions in the code would ensure such read/write-through, but merely as a side effect - they also lock the bus for every instruction, what's not required when concurrent access has been excluded by other means before. Conclusion: We need a documentation of the FPC specific means of cache synchronization, with their guaranteed effects on every target[1]. Furthermore we need concrete examples[2], how (to what extent) it's required to use these special instructions/procedures, in examples like above. When cache synchronization is a big issue, then the usage of related (thread-unaware) objects should be discussed as well, i.e. how to ensure that their use will cause no trouble, e.g. by invalidating the entire cache before. [1] When the effects of the primitíves vary amongst targets, then either more specific documentation is required, or higher level target-insensitive procedures with a guaranteed behaviour. Eventually both, so that the experienced coder can use conditional code for the different handling on various targets. [2] References to e.g. C code samples IMO is inappropriate, because the user cannot know what special handling such a different compiler will apply to its compiled code (see volatile). DoDi
Re: [fpc-devel] volatile variables
On 06/29/2011 09:00 PM, Vinzent Höfler wrote: POSIX: pthread_mutex_(un)lock() Co. Or, maybe I didn't understand the question... I suppose, you did understand what I intended to say. Regarding FPC, TCriticalSection is a decent encapsulation for pthread_mutex_... when used in this way. But e.g. if you use a TThreadList instance myList with multiple threads it can't be the way to go to include any occurrence of myList.xxx by a critical section just to make sure that the cache synchronization has happened and the instance variable itself in fact is valid. Moreover I don't think it can be the way to go to enforce the application programmer to plan his software in a way that somehow else he needs to make sure that such an instance variable arrives in the appropriate CPU's cache. So I do hope that the constructors of the hardware, the OS and the libraries already somehow did take care of this issue O:-) , -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 06/29/2011 09:44 PM, Vinzent Höfler wrote: On Wed, 29 Jun 2011 13:28:20 +0200, Hans-Peter Diettrich Ada2005 RM: |C.6(16): For a volatile object all reads and updates of the object as | a whole are performed directly to memory. |C.6(20): The external effect [...] is defined to include each read and | update of a volatile or atomic object. The implementation shall | not generate any memory reads or updates of atomic or volatile | objects other than those specified by the program. That's Ada's definition of volatile. C's definition is less stronger, but should basically have the same effect. Nice and what is the FPC definition ? Thanks, -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
30.06.2011 13:31, Hans-Peter Diettrich: If so, would it help to enclose above instructions in e.g. Synchronized begin update the links... end; If by such hypothetical synchronized operator you mean just memory barriers and nothing else, then AFAICS this would not be of much use in practice, because in case of strictly one thread barriers are unneeded anyway, and in case of 2 or more threads barriers are typically insufficient (because most of the time threads would also need race protection, which is not provided by barriers alone). If what you mean here is rather a full-blown critical section with just some more elegant and generic syntax (i.e. a replacement of EnterCriticalSection/LeaveCriticalSection a la java), then yes, it might be nice, but mostly aesthetic-wise. Nikolai ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 30 Jun 2011, at 10:42, Michael Schnell wrote: On 06/29/2011 09:44 PM, Vinzent Höfler wrote: That's Ada's definition of volatile. C's definition is less stronger, but should basically have the same effect. Nice and what is the FPC definition ? There is none. FPC has a volatile modifier in svn trunk, but it currently only affects the node tree optimizer. There are many other parts of the compiler that completely ignore it (from the inliner to the i386 assembler peephole optimizer), and hence it is probably not that useful in its current incarnation. If such functionality were properly implemented, the final behaviour and definition would however probably be pretty much the same as the ADA definition Vinzent posted. However, as has been mentioned several times before: volatile has absolutely no use to make programs thread safe. Its only use is for memory mapped I/O. Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
Mark Morgan Lloyd wrote: Mark Morgan Lloyd wrote: Mark Morgan Lloyd wrote: Felipe and Michael have pointed out to me that David Zhang's mipsel port has been integrated into trunk. Before I start trying to build it, could anybody comment on how complete an implementation it is? Apart from that I can confirm that David's code on Sourceforge is good enough for trivial programs that run on a recent Linux hosted by Qemu. Current situation is that after ensuring the existence of dummy files I've stepped back from the RTL code and am looking at a problem that appears to be caused by floating point initialisation in the cross compiler. A define incorrectly suggested that MIPS was using MM registers. Now back following missing files/symlinks in the RTL. http://wiki.lazarus.freepascal.org/Native_MIPS_Systems#Stall_point I hope to have an SGI machine in a few days, provided that it works (it's been stored in a garage) I might be able to look at mips as well as mipsel. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
In our previous episode, Mark Morgan Lloyd said: Current situation is that after ensuring the existence of dummy files I've stepped back from the RTL code and am looking at a problem that appears to be caused by floating point initialisation in the cross compiler. A define incorrectly suggested that MIPS was using MM registers. Now back following missing files/symlinks in the RTL. http://wiki.lazarus.freepascal.org/Native_MIPS_Systems#Stall_point wrt the missing of STAT.inc, IIRC the stat record definition on linux was found to be slightly architecture dependant (x86 being a bit different due to legacy reasons, of both FPC and Linux). Probably the wisest is to take the PPC one and check it quickly against the header. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On Thu, Jun 30, 2011 at 4:31 AM, Hans-Peter Diettrich drdiettri...@aol.com wrote: After these considerations I'd understand that using Interlocked instructions in the code would ensure such read/write-through, but merely as a side effect - they also lock the bus for every instruction, what's not required when concurrent access has been excluded by other means before. I too have come to that conclusion. It remains to be seen/decided though, who's responsibility is it to ensure/enforce coherency? Jonas was suggesting that the posix implementation (but he probably meant kernel) was already doing that via CriticalSection. Furthermore we need concrete examples[2], how (to what extent) it's required to use these special instructions/procedures, in examples like above. When cache synchronization is a big issue, then the usage of related (thread-unaware) objects should be discussed as well, i.e. how to ensure that their use will cause no trouble, e.g. by invalidating the entire cache before. Some FPC work may have been done in this area - intentional or not. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 06/30/2011 11:52 AM, Jonas Maebe wrote: On 30 Jun 2011, at 10:38, Michael Schnell wrote: Regarding FPC, TCriticalSection is a decent encapsulation for pthread_mutex_... when used in this way. But e.g. if you use a TThreadList instance myList with multiple threads it can't be the way to go to include any occurrence of myList.xxx by a critical section just to make sure that the cache synchronization has happened and the instance variable itself in fact is valid. That's where optimization comes in, Very wrong assumption. Optimization is not the question here it's just working vs. not working. In a previous post the claim was that if the variable is not in the cache of the appropriate CPU, a cache synchronization is not automatically done by the underling system (Hardware / OS / Libraries) and thus the variable has an erroneous value unless the application program uses some special means to provide coherency. In fact I doubt this, but this was the claim and thus needs to be discussed here. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
Marco van de Voort wrote: In our previous episode, Mark Morgan Lloyd said: Current situation is that after ensuring the existence of dummy files I've stepped back from the RTL code and am looking at a problem that appears to be caused by floating point initialisation in the cross compiler. A define incorrectly suggested that MIPS was using MM registers. Now back following missing files/symlinks in the RTL. http://wiki.lazarus.freepascal.org/Native_MIPS_Systems#Stall_point wrt the missing of STAT.inc, IIRC the stat record definition on linux was found to be slightly architecture dependant (x86 being a bit different due to legacy reasons, of both FPC and Linux). Probably the wisest is to take the PPC one and check it quickly against the header. Thanks, I'm already using ppc for comparison purposes which is why I had a hard time finding the MM issue :-) Question if I may: should I be avoiding symlinks as non-portable? I'm gradually hacking through with a mixture of command-line compilation and using Lazarus to step through. I considered looking at FPP to see if I could log all procedure calls, I thought I was looking for something that hadn't been initialised properly. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 06/29/2011 09:44 PM, Vinzent Höfler wrote: If they are accessed by only one thread, I'd assert that each core's view on its own cache is not susceptible to memory ordering issues I don't suppose this is that simple. AFAIK, the cache does not work on byte addresses, but on entities of cache lines that are 128 byte or whatever. So a cache can't see if a certain variable is accessed by another thread, but needs to store and reload a complete line. Thus one CPU might modify a value and write back the cache line while another CPU modified a value of another variable in the same cache line. As this situation obviously is handled by the hardware, I do suppose that the case of multiple CPUs modifying the same variable is handled, too (but of course without software considerations not regarding the hardcore cases like an atomic read-modify-write access or cycle-level memory ordering). -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 06/30/2011 11:45 AM, Jonas Maebe wrote: There is none. FPC has a volatile modifier in svn trunk, but it currently only affects the node tree optimizer. ... Its only use is for memory mapped I/O. I don't suppose the node tree optimizer is memory mapped I/O ??? -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
RE: [fpc-devel] MIPS
I had a MIS related question:
using fullcycle I got a warinng about
defdynlinker not being initialized fro mips in systems/t_linux.pas unit.
Is this a correct fix,
Index: t_linux.pas
===
--- t_linux.pas (révision 17859)
+++ t_linux.pas (copie de travail)
@@ -192,6 +192,9 @@
{$endif FPC_ARMEL}
{$endif arm}
+{$ifdef mips}
+ defdynlinker:='/lib/ld-linux.so';
+{$endif mips}
{
Search order:
glibc 2.1+
or can somoeone tell us what the default linker is for a mips linux
machine?
Pierre Muller
-Message d'origine-
De : fpc-devel-boun...@lists.freepascal.org [mailto:fpc-devel-
boun...@lists.freepascal.org] De la part de Mark Morgan Lloyd
Envoyé : jeudi 30 juin 2011 14:40
À : fpc-devel@lists.freepascal.org
Objet : Re: [fpc-devel] MIPS
Marco van de Voort wrote:
In our previous episode, Mark Morgan Lloyd said:
Current situation is that after ensuring the existence of dummy files
I've stepped back from the RTL code and am looking at a problem that
appears to be caused by floating point initialisation in the cross
compiler.
A define incorrectly suggested that MIPS was using MM registers. Now
back following missing files/symlinks in the RTL.
http://wiki.lazarus.freepascal.org/Native_MIPS_Systems#Stall_point
wrt the missing of STAT.inc, IIRC the stat record definition on linux
was
found to be slightly architecture dependant (x86 being a bit different
due
to legacy reasons, of both FPC and Linux).
Probably the wisest is to take the PPC one and check it quickly against
the
header.
Thanks, I'm already using ppc for comparison purposes which is why I had
a hard time finding the MM issue :-)
Question if I may: should I be avoiding symlinks as non-portable?
I'm gradually hacking through with a mixture of command-line compilation
and using Lazarus to step through. I considered looking at FPP to see if
I could log all procedure calls, I thought I was looking for something
that hadn't been initialised properly.
--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk
[Opinions above are the author's, not those of his employers or
colleagues]
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 06/30/2011 11:31 AM, Hans-Peter Diettrich wrote: Consider the shareable bi-linked list, where insertion requires code like this: list.Lock; //prevent concurrent access ... //determine affected list elements new.prev := prev; //prev must be guaranteed to be valid new.next := next; prev.next := new; next.prev := new; list.Unlock; I really would like to see a test case proving my assumption wrong that this in fact just woks We found that - my original volatile question is invalid, as the function calls done with list.lock and list.unlock are a volatile barrier preventing the compiler from caching some value inappropriately in a register, simply because they are function calls. - the MUTEX operations done with list.lock and list.unlock (at least on PCs in Linux and Windows) use library calls that includes memory barriers - if the potential cache incoherency would not be handled by Hardware / OS / Libraries on behalf of user land programs, I feel that this would so disastrous and ubiquitous that it result in so many programs not working on SMP systems that it would be a really well known issue. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
In our previous episode, Mark Morgan Lloyd said: Probably the wisest is to take the PPC one and check it quickly against the header. Thanks, I'm already using ppc for comparison purposes which is why I had a hard time finding the MM issue :-) Question if I may: should I be avoiding symlinks as non-portable? For anything that should go back into SVN: no symlinks (*). (and in general, even without SVN concerns, I don't think heavy symlinking is a good concept in sources. This would cause only problems because sb might modify it for ppc, and not notice mips also linked to the file) (*)http://subversion.apache.org/faq.html#symlinks ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
On 06/30/2011 02:40 PM, Mark Morgan Lloyd wrote: Question if I may: should I be avoiding symlinks as non-portable? NTFS can do symlinks even though the normal M$ tools are not able to manage them. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On Thu, Jun 30, 2011 at 8:04 AM, Michael Schnell mschn...@lumino.de wrote: - if the potential cache incoherency would not be handled by Hardware / OS / Libraries on behalf of user land programs, I feel that this would so disastrous and ubiquitous that it result in so many programs not working on SMP systems that it would be a really well known issue. I agree. This problem when encountered was hidden during steady state testing. The list was consistently working as expected. Where it failed was when the core was ~80% usage and it walked - and it was on my test system here using daily Ubuntu x64 builds - lord knows what kernel it had those days. Infact, it could possibly be an ubiquitous exploit waiting for disaster when code normally running code encounters stale values across cores. You can imagine systems designed to never fail - fail for no known reason at all... In a case I observed, it did cause a significant problem to the server. Yes, it was disastrous, and ONLY evident during stress tests. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 06/30/2011 03:29 PM, Andrew Brunner wrote: In a case I observed, it did cause a significant problem to the server. Yes, it was disastrous, and ONLY evident during stress tests. ... which encourages me to suggest that it is a nasty bug _somewhere_. Maybe even in the hardware used. I can't believe that the Posix specs allow for such problem to occur. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
Am 30.06.2011 14:53, schrieb Michael Schnell: On 06/30/2011 11:45 AM, Jonas Maebe wrote: There is none. FPC has a volatile modifier in svn trunk, but it currently only affects the node tree optimizer. ... Its only use is for memory mapped I/O. I don't suppose the node tree optimizer is memory mapped I/O ??? No, but the node tree optimizer might optimize away two consecutive reads from or writes to a memory mapped variable which might be necessary for the hardware... e.g. (theoretical pseudo code scenario) === source begin === var a: Integer; volatile; absolute $42; b: Integer; begin b := a; // this might be necessary for the hardware, but the optimizer will remove the first statement if it doesn't know about volatile b := a; end; === source end === Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Safely shareable objects
Preface: I don't want to extend the current discussions, about threads etc. The subject here is a possible language extension, that might allow to simplify the construction of shareable objects, while improving performance vs. already possible workarounds, optionally adding means to allow the compiler to assure correct and safe use of such objects. The idea is a merge of reference counting and thread synchronization, which I could not find in any other language yet (dunno about ADA in detail), but which can be implemented easily in OPL/FPC. Inspiration came from TThreadList and Interfaces. When interface references are reference counted, then it's possible to use the RefCount for automatic locking of the related object, so that it can be used by multiple threads *only* sequentially - preventing race conditions. Until here everything can be achieved in every class, e.g. derived from TInterfacedObject, by modifying _AddRef and _Release accordingly. But the use of the object via interface references slows down the entire procedure, due to the overhead involved in calling interface methods vs. object methods, and the maintenance of the additional interface declaration. I have no idea about the practical impact of that overhead, so the only questions are: 1. Would it make sense, perfomance-wise, to add reference-counted classes as a new language element? 2. Could further support be added, so that the compiler can assure that references to such objects can not be stored outside the calling procedure? I.e. only local variables and parameters would be allowed to contain such references, because otherwise the object could be locked permanently. Let me give an example, based on TThreadList. Such a list is intended to hold objects, that are for exclusive use by only one thread at the same time. The disadvantage is the locking of *all* list elements at the same time, what can either cure or create possible deadlocks. Now you may understand why I would like to have single objects, protected in a similar way. Also other organizations (pipe=FIFO, stack=LIFO...) may be desireable with *shareable objects*, i.e. objects that are *intended* for communicating data safely across thread boundaries. Based on already available features, an IShareable interface could be designed, whose implementation must lock the object when its RefCount exceeds 1, and unlocks it again when the RefCount reaches 1 again. You may note the weak point: every class can implement that interface differently, possibly violating the implied rules. This could not happen when a refcounted *class* could not override the implementation in the base class. So let's assume that all this is implemented in a TShareableObject class, for use similar to TInterfacedObject, where nobody has to care about the implementation of the required interface-related (and here: thread/data synchronization related) methods. Now we can assume that IShareable objects/references can be used all over the code, with no further precautions in explicit code required, that the use of the object *and* its content is always synchronized and sequentialized properly. Specific interfaces can be derived from IShareable, e.g. IShareableList, IShareableParamsAndResultfor_whatever_purpose. As far as FPC is concerned, the elimination of the additional interfaces in above workaround IMO would not only increase the acceptance of this model in the design of threaded applications, but also could demonstrate that Pascal rules!, increasing the acceptance of FPC/OPL as an outstanding compiler and language :-) Am I too euphoric? DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Safely shareable objects
To me this seems like an excellent idea. Especially when combined with features like parallel loop and future variables (like existing in Delphi Prism), that can be implemented based on a thread pool. All this can increase performance on SMP systems by invisibly using threads. -Michael ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] volatile variables
On 30 Jun 2011, at 14:26, Michael Schnell wrote: On 06/30/2011 11:52 AM, Jonas Maebe wrote: On 30 Jun 2011, at 10:38, Michael Schnell wrote: But e.g. if you use a TThreadList instance myList with multiple threads it can't be the way to go to include any occurrence of myList.xxx by a critical section just to make sure that the cache synchronization has happened and the instance variable itself in fact is valid. That's where optimization comes in, Very wrong assumption. I simply misunderstood you (I thought you meant that there should be a solution to avoid having to use critical sections around all accesses to fields that may be accessed from multiple threads). However, what is a wrong assumption on your part is that when I replied to fpc-other with you in CC, explicitly mentioning moved to fpc-other because unrelated to developing FPC, that it's ok to reply back to fpc-devel (moreover removing that remark in the process). This thread has gone off topic for long enough now on this list, and further posts on this topic to this list will be rejected. Jonas FPC mailing lists admin ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] fpc build problems on some debian armel buildds
Freepascal has been failing sometimes with einouterror file not found on armel buildds but i'm not really seeing any pattern to the failures either in terms of upstream versions or in terms of buildds 2.4.0-2 was successfully built on muscat 2.4.0-3 failed on arnold 2.4.0-4 failed on arnold and ancina (the latter being a retry at my request) 2.4.2-1 was sucessfully built on alwyn 2.4.2-2 and 2.4.2-2 were successfully build on ancina 2.4.4-1 failed on antheil The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. This is worrying for me because I use freepascal on armel and yet it's pretty hard to find a fix for a bug without a meaningful error message and which I cannot reproduce. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? Could some arm users try building fpc and reporting back if they can reproduce the error and if so on what hardware? (note: you must build with -B or you will run into an unrelated failure). If it fails for you can you try running the failing command under strace to try and determine what the file was? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Re: fpc build problems on some debian armel buildds
On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular apt-get upgrades to get latest toolchain etc in sid. Riku ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Safely shareable objects
On Thu, Jun 30, 2011 at 11:34 AM, Michael Schnell mschn...@lumino.de wrote: To me this seems like an excellent idea. +1 Marcos Douglas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
[fpc-devel] Re: fpc build problems on some debian armel buildds
On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: Freepascal has been failing sometimes with einouterror file not found on armel buildds but i'm not really seeing any pattern to the failures either in terms of upstream versions or in terms of buildds 2.4.0-2 was successfully built on muscat 2.4.0-3 failed on arnold 2.4.0-4 failed on arnold and ancina (the latter being a retry at my request) 2.4.2-1 was sucessfully built on alwyn 2.4.2-2 and 2.4.2-2 were successfully build on ancina 2.4.4-1 failed on antheil The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. This is worrying for me because I use freepascal on armel and yet it's pretty hard to find a fix for a bug without a meaningful error message and which I cannot reproduce. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? muscat is a Thecus N2100, (iop32x with 512MB RAM) alwyn and antheil are identical Marvell development boards - see http://blog.einval.com/2010/09/27#marvell_buildds for more details. If you want to know more, http://db.debian.org/machines.cgi lists more details for most of the official machines used by Debian. If you're getting inconsistent results between antheil and alwyn then that's worrying. :-( You can ask for access to abel (the armel porter box) and play with stuff there if it helps. -- Steve McIntyre, Cambridge, UK.st...@einval.com liw everything I know about UK hotels I learned from Fawlty Towers ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Safely shareable objects
Hello FPC, Thursday, June 30, 2011, 4:07:22 PM, you wrote: HPD Now we can assume that IShareable objects/references can be used all HPD over the code, with no further precautions in explicit code required, HPD that the use of the object *and* its content is always synchronized and HPD sequentialized properly. Specific interfaces can be derived from HPD IShareable, e.g. IShareableList, HPD IShareableParamsAndResultfor_whatever_purpose. So you mean that something like: IMyObject inherited from IShareable the compiler must add stub code on each function/procedure/property to lock the interface ? So a function like: function iMyObject.Increment(a: integer): integer; begin a:=a+1; end; must be silently modified by the compiler to something like: function iMyObject.Increment(a: integer): integer; begin IObjectCritical.Enter; try a:=a+1; finally IObjectCritical.Leave; end; end; ? To me it looks extremelly overkill. If you think in the other approach, so when interface count is higher than one enter the critical section, it could not be done IMHO because when refcount reach 2 it could happend in thread 2 while thread 1 is inside a procedure of the object which could lead to unexpected results. Maybe I missed something ? -- Best regards, José ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
Pierre Free Pascal wrote:
I had a MIS related question:
using fullcycle I got a warinng about
defdynlinker not being initialized fro mips in systems/t_linux.pas unit.
Is this a correct fix,
Index: t_linux.pas
===
--- t_linux.pas (révision 17859)
+++ t_linux.pas (copie de travail)
@@ -192,6 +192,9 @@
{$endif FPC_ARMEL}
{$endif arm}
+{$ifdef mips}
+ defdynlinker:='/lib/ld-linux.so';
+{$endif mips}
{
Search order:
glibc 2.1+
or can somoeone tell us what the default linker is for a mips linux
machine?
I can't say at this point. I'm working through step-by-step starting
with a cross compiler, i.e. I'm a long way from having a full build.
/If/ I get to the point of a full build then hopefully somebody will
review my work and consider merging it with trunk. Until then I'm
working with a local copy, but the issues I find are noted on the wiki page.
--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk
[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
Marco van de Voort wrote: Question if I may: should I be avoiding symlinks as non-portable? For anything that should go back into SVN: no symlinks (*). (and in general, even without SVN concerns, I don't think heavy symlinking is a good concept in sources. This would cause only problems because sb might modify it for ppc, and not notice mips also linked to the file) (*)http://subversion.apache.org/faq.html#symlinks Thanks, noted. Obviously while I've just got a local copy on my system I can easily locate symlinks, SVN is very much in the future since I've got limited confidence in my work so far. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
On Thu, Jun 30, 2011 at 6:01 PM, Mark Morgan Lloyd markmll.fpc-de...@telemetry.co.uk wrote: or can somoeone tell us what the default linker is for a mips linux machine? I can't say at this point. I'm working through step-by-step starting with a cross compiler, i.e. I'm a long way from having a full build. To answer his question you only need to issue the following command in your MIPS emulator running Linux: which ld This should tell you where the linker is -- Felipe Monteiro de Carvalho ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: fpc build problems on some debian armel buildds
Riku Voipio wrote: On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular apt-get upgrades to get latest toolchain etc in sid. There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year: -8- On 05 Oct 2010, at 10:05, Mark Morgan Lloyd wrote: When running 2.4.0 on an ARM system (Debian v5 Lenny, armel) with limited memory (32Mb RAM + 768Mb swap) and using it to compile a large project (Lazarus 0.9.28.2) I'm seeing intermittent failures which go away if the make is restarted. I've not seen this running on other platforms, and I don't believe it is a problem in the Lazarus sources since the build will eventually complete giving me runnable code. A couple of days ago I fixed an error in svn trunk for ARMEL that caused the stack to become temporarily unbalanced after performing syscalls with 5 or more parameters (the bug is still there for OABI, but I can't fix that because I don't have access to an OABI machine). A side-effect of that bug was that if the caller passed the address of its own result as one of the parameters to the system call, it would afterwards return a random value as its result and checks for error results caused random failures like the one you posted (the reproducible case that allowed me to fix it was a similar error). Jonas -8- There might be alignment issues on ARM and SPARC but I've only seen those with Lazarus, not with FPC itself. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: fpc build problems on some debian armel buildds
Riku Voipio wrote: Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. Do you know specifically how the debian buildds handle unaligned accesses? do they fix them up? do they generate a bus error? do they let them go silently wrong? Regardless though I don't think that is the issue here as I've finally seen the einouterror in qemu after trying building in the precise path the buildd used (and I managed to reproduce it twice in a row under those circumstances). Unfortunately re-running the failing command manually succeeds so it seems like the long path alone is insufficiant to trigger the bug and there must be something in the environment the build system provides as well. I tried shoving a strace in there but that also seemed to stop it failing so i've no idea how I can track this down further. Another complication is that this is happening as the first new compiler is being built using the compiler installed on the build system, in other words even if the issue is tracked down and fixed a new manual upload is likely to be needed to fix it. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: fpc build problems on some debian armel buildds
Mark Morgan Lloyd wrote: Riku Voipio wrote: On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular apt-get upgrades to get latest toolchain etc in sid. There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year: Do you know if that fix has made it into any stable releases and if so which ones? -8- On 05 Oct 2010, at 10:05, Mark Morgan Lloyd wrote: When running 2.4.0 on an ARM system (Debian v5 Lenny, armel) with limited memory (32Mb RAM + 768Mb swap) and using it to compile a large project (Lazarus 0.9.28.2) I'm seeing intermittent failures which go away if the make is restarted. I've not seen this running on other platforms, and I don't believe it is a problem in the Lazarus sources since the build will eventually complete giving me runnable code. A couple of days ago I fixed an error in svn trunk for ARMEL that caused the stack to become temporarily unbalanced after performing syscalls with 5 or more parameters (the bug is still there for OABI, but I can't fix that because I don't have access to an OABI machine). A side-effect of that bug was that if the caller passed the address of its own result as one of the parameters to the system call, it would afterwards return a random value as its result and checks for error results caused random failures like the one you posted (the reproducible case that allowed me to fix it was a similar error). Jonas -8- There might be alignment issues on ARM and SPARC but I've only seen those with Lazarus, not with FPC itself. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
Felipe Monteiro de Carvalho wrote:
On Thu, Jun 30, 2011 at 6:01 PM, Mark Morgan Lloyd
markmll.fpc-de...@telemetry.co.uk wrote:
or can somoeone tell us what the default linker is for a mips linux
machine?
I can't say at this point. I'm working through step-by-step starting with a
cross compiler, i.e. I'm a long way from having a full build.
To answer his question you only need to issue the following command in
your MIPS emulator running Linux:
which ld
This should tell you where the linker is
I think his is a loader issue- the binary needs to know what .so to pull
in that later allows it to work out paths etc.
+{$ifdef mips}
+ defdynlinker:='/lib/ld-linux.so';
+{$endif mips}
--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk
[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: fpc build problems on some debian armel buildds
On 30 Jun 2011, at 19:04, peter green wrote: Mark Morgan Lloyd wrote: Riku Voipio wrote: On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular apt-get upgrades to get latest toolchain etc in sid. There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year: Do you know if that fix has made it into any stable releases and if so which ones? It hasn't. It's only available in svn trunk (r16073) Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
On Thu, Jun 30, 2011 at 7:05 PM, Mark Morgan Lloyd markmll.fpc-de...@telemetry.co.uk wrote: I think his is a loader issue- the binary needs to know what .so to pull in that later allows it to work out paths etc. ops, it seams that you are right =O -- Felipe Monteiro de Carvalho ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Safely shareable objects
José Mejuto schrieb: So you mean that something like: IMyObject inherited from IShareable the compiler must add stub code on each function/procedure/property to lock the interface ? No. The object is locked as long the code holds the reference to it. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
In our previous episode, Mark Morgan Lloyd said: I think his is a loader issue- the binary needs to know what .so to pull in that later allows it to work out paths etc. Indeed. ldd ./gccbinary name might be more revealing. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re[2]: [fpc-devel] Safely shareable objects
Hello FPC, Thursday, June 30, 2011, 9:42:36 PM, you wrote: HPD José Mejuto schrieb: So you mean that something like: IMyObject inherited from IShareable the compiler must add stub code on each function/procedure/property to lock the interface ? HPD No. The object is locked as long the code holds the reference to it. In that case read my note at the end of the e-mail, I think it could not be done in that way. -- Best regards, José ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] MIPS
Felipe Monteiro de Carvalho wrote:
On Thu, Jun 30, 2011 at 7:05 PM, Mark Morgan Lloyd
markmll.fpc-de...@telemetry.co.uk wrote:
I think his is a loader issue- the binary needs to know what .so to pull in
that later allows it to work out paths etc.
ops, it seams that you are right =O
I only spotted it when I started wondering whether 'which' was going to
be any use in a cross-development environment such as I think one has to
start with in this case.
I presume the answer to the original question is use the one on the
system you're working on. In other words, if building a cross compiler
to run on i386 assume /lib/ld-linux.so.2; the {$ifdef mips} will only
come into play when building to run natively.
--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk
[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: fpc build problems on some debian armel buildds
peter green wrote: Riku Voipio wrote: Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. Do you know specifically how the debian buildds handle unaligned accesses? do they fix them up? do they generate a bus error? do they let them go silently wrong? Google suggests that it's a known issue [1], and that the degree of warning etc. may be controlled using /proc/cpu/alignment [2] 1] http://qemu-forum.ipi.fi/viewtopic.php?f=20t=5543 2] http://lecs.cs.ucla.edu/wiki/index.php/XScale_alignment#Have_the_kernel_find_the_problem_for_you -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Re: fpc build problems on some debian armel buildds
Jonas Maebe wrote: On 30 Jun 2011, at 19:04, peter green wrote: Mark Morgan Lloyd wrote: Riku Voipio wrote: On Thu, Jun 30, 2011 at 03:33:36PM +0100, peter green wrote: The current FPC package builds fine in qemu for me. I will attempt to try on some real hardware too when I get a chance. Qemu allows unaligned memory accesses, which do not always work on real hardware, especially on armv5 and other older arms. Is there a list anywhere of what hardware the buildds run on and/or any other interesting information about their setups so I can try to figure out what if anything the failing buildds have in common. Has anything changed in ancina's configuration recently? All currently running armel buildd's are identical marvell mv78x00 boards. Updates on ancina are the regular apt-get upgrades to get latest toolchain etc in sid. There were issues with some versions of FPC, related to more than a certain number of parameters (four?) being passed. Jonas wrote the following on the 5th October last year: Do you know if that fix has made it into any stable releases and if so which ones? It hasn't. It's only available in svn trunk (r16073) Is it reasonablly backportable? Are there any plans for a new stable release any time soon either based on the current stable release branch or freshly branched from trunk? Jonas___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Safely shareable objects
José Mejuto schrieb: IMyObject inherited from IShareable the compiler must add stub code on each function/procedure/property to lock the interface ? HPD No. The object is locked as long the code holds the reference to it. In that case read my note at the end of the e-mail, I think it could not be done in that way. No other thread can have a reference to the same object, and consequently cannot call its methods. _AddRef: if RefCount 0 then EnterCriticalSection(myCS); This will block any other thread in the attempt to get another reference, until the thread owning myCS has left the critical section. DoDi ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
