Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-07 Thread Sven Barth

On 06.03.2013 14:36, Sven Barth wrote:

Am 06.03.2013 14:29, schrieb Michael Schnell:

On 03/06/2013 01:43 PM, Sven Barth wrote:



And how does one the current TThread?

Since a few weeks: TThread.CurrentThread ;)



This does in fact use a threadvar:

threadvar
  { the instance of the current thread; in case of an external thread
this is
Nil until TThread.GetCurrentThread was called once (the RTLs need
to ensure
that threadvars are initialized with 0!) }
  CurrentThreadVar: TThread;


and (unless the compiler optimizes this out) even accesses it twice:


  if not Assigned(CurrentThreadVar) then
CurrentThreadVar := TExternalThread.Create;
  Result := CurrentThreadVar;


From within the TThread object, simply using Self seems more
appropriate unless there is a chance that the same TThread instance is
used for multiple OS-Threads. I don't know if/how this is possible.

From outside I feel that AnyThread.GetCurrentThread does not make
much senses.

When doing TThread.GetCurrentThread as a class function I think I
should get self of same when I am in the code that is called from
Execute of some TThread instance. I  don't see what I want to see
when I'm not.

The code seems to try to avoid the case that a no TThread instance
when  GetCurrentThread is called as a a class function. I don't know
if/how this is sensible.

Using CurrentThread only seems sensible within the the code of a
component that has been called by the code of a TThread instance. But
here using an appropriate back-link property can easily be used to
avoid accessing the threadvar. (or using  CurrentThread once to set a
property  and then just accessing same.) But this of course needs to
be done in user code and the RTL can't force it.


The TThread.CurrentThread is mainly for access in functions that don't
get passed a TThread instance and to also get a TThread instance for
threads not created by the RTL (the TExternalThread.Create line
above). Don't forget that TThread.CurrentThread is a class
property/function, so you can't access Self.

We could optimize it like this:

=== code begin ===

Result := CurrentThreadVar;
if not Assigned(Result) then begin
   Result := TExternalThread.Create;
   CurrentThreadVar := Result;
end;

=== code end ===

So that in the normal case only one access to the threadvar is used.


I've implemented this optimization in r23706. So in the normal case the 
threadvar is only read once and in the worst case it's read once and 
written once.


Regards,
Sven

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-07 Thread Graeme Geldenhuys
Nice... In your last message: 69 lines of quotes, 4 levels deep, and 3
lines for the actual message. Keep up the good work.

Netiquette?

G.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-07 Thread Sven Barth
Am 08.03.2013 00:43 schrieb Graeme Geldenhuys gra...@geldenhuys.co.uk:

 Nice... In your last message: 69 lines of quotes, 4 levels deep, and 3
 lines for the actual message. Keep up the good work.

 Netiquette?

This should have been shortened indeed...

To my defense: I wanted to get this mail out, fix another problem (see
fpc-pascal list) and hop into bed...
You should also take into consideration that I normally *do* shorten my
mails.

Regards,
Sven
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-07 Thread Vincent Snijders
2013/3/8 Graeme Geldenhuys gra...@geldenhuys.co.uk:
 Nice... In your last message: 69 lines of quotes, 4 levels deep, and 3
 lines for the actual message. Keep up the good work.

Good quoting, it just read that message and got a comprehensive story,
especially because my autofilter deletes mails from Michael Schnelll.


 Netiquette?

Spoilt by gmail.

Vincent
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Daniël Mantione



Op Tue, 5 Mar 2013, schreef Henry Vermaak:


Damn.  My custom config kernel compiles stable kernels in 3-5 minutes on
a quad core Xeon, which isn't bad.  Did you build with the standard
config?


What is the standard config? As the operating system is Scientific Linux 
6, I'm using the config for the standard 2.6.32 kernel with as little 
modifications as possible to use it for 3.3.2.


I don't have a 4 core Xeon system, but I tried to simulate it on a dual 
Xeon 2670 system by only using 4 cores:


[root@node037 src]# time numactl --physcpubind=0-3 make -j 4

...

real11m45.748s
user38m30.405s
sys 4m48.634s


Daniël___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Michael Schnell

On 03/04/2013 12:47 PM, Michael Schnell wrote:


In all fpc enabled patforms I examined (X86 Windows, X86 Linux, X64-64 
Windows, X86-64 Linux, ARM Linux (32 Bits) the platform provides as 
well library calls as dedicated threadvar pointer registers. In 
windows the registers seem to be undocumented but obviously stable, 
as a change would break a lot existing C/C++ software.


All C/C++ compilers I examined used the threadvar pointer Registers 
for speed..




FWIW:
I rechecked this and can add these results:

With ARM, the pre-Cortex Arch on Linux uses A9 as a pointer to the TLS 
(Thread local Storage http://en.wikipedia.org/wiki/Thread-local_storage 
, that holds the threadvars). So (as with all x86 variants) no library 
or system call is necessary.


With Cortex, this has changed to the use of the CP15 Register (freeing 
A9 to allow for better optimization). Now, CP15 is in fact not very 
suitable for this purpose, as it can only be accessed with privileged 
instructions and thus not in user mode (where the TLS in fact is 
invented for). Thus a system call (or trap) is needed to get the address 
of a threadvar. As a system call supposedly is rather costly on ARM  
(invalidating the cache, as here, (other than with x86) the cache is 
closer to the CPU than the MMU) using a library call will not do 
additional harm.





Generally:

The frequent access to threadvars is necessary with threaded programs 
that use the same code in multiple threads to allow for each thread to 
find easily out who I am even in a deeply nested function environment.


But in an object driven language, this can easily be overcome by 
creating an instance of an object for each thread and with that, you 
always have the Self pointer as a thread specific entity. Happily 
Self resides on the stack that is dedicated for each thread, anyway, 
and is created automatically for any procedure or object. So the use 
of any threadvar is necessary only when creating the thread-related 
object but it's not necessary to access any threadvar frequently.





Conclusion:

With FPC, the optimization of threadvar access is not necessary. Instead 
it is highly recommended to always use thread specific instances of the 
classes that do the threaded work. TThread of course provides this out 
of the box.


-Michael

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Florian Klaempfl

Am 06.03.2013 11:12, schrieb Michael Schnell:

So (as with all x86 variants) no library
or system call is necessary.


Wrong. This is not true in the general case, please read the appropriate 
api documents.




With Cortex, this has changed to the use of the CP15 Register (freeing
A9 to allow for better optimization). Now, CP15 is in fact not very
suitable for this purpose, as it can only be accessed with privileged
instructions and thus not in user mode (where the TLS in fact is
invented for).Thus a system call (or trap) is needed to get the address
of a threadvar.


Wrong. On linux this can be done by a jump to a certain adress.


Conclusion:

With FPC, the optimization of threadvar access is not necessary. Instead
it is highly recommended to always use thread specific instances of the
classes that do the threaded work. TThread of course provides this out
of the box.



And how does one the current TThread?

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Michael Schnell

On 03/06/2013 11:45 AM, Florian Klaempfl wrote:

So (as with all x86 variants) no library
or system call is necessary.


Wrong. This is not true in the general case, please read the 
appropriate api documents.
I do know that on Windows it's documented like this by Microsoft. But I 
also know that the Microsoft C compiler (and AFAIK, gnu as well) does 
not adhere to that documentation but simply uses the appropriate 
selector register that the Intel chip designers happily provided for 
such kind of purpose. (We already discussed this here some years ago.)


Wrong. On linux this can be done by a jump to a certain adress.
OK. I did not thoroughly enough check into this. Does the jump to the 
system region automatically provide the mode switch without changing 
anything in the MMU (and this without the necessity to invalid the cache) ?


If yes a glibc library call might provide a noticeable overhead and it 
might be advantageous to do the system call directly in the RTL.




And how does one the current TThread?


Do you mean something like this:


Type
TMyThread = class(TThread)
 private
  currentthread: Integer;
end.

var
  threads: array[1..100] of TMyThread;

for i := 1 to 100 do begin
  thread[i] := TMyThread.Create;
  thread[i].currentthread := i;
end;


TMyThread.Execute;
var
  s: String;
begin
  s := 'I am ' + InToString(currentthread);
end;

?
-Michael
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Henry Vermaak
On Wed, Mar 06, 2013 at 10:50:10AM +0100, Daniël Mantione wrote:
 
 
 Op Tue, 5 Mar 2013, schreef Henry Vermaak:
 
 Damn.  My custom config kernel compiles stable kernels in 3-5 minutes on
 a quad core Xeon, which isn't bad.  Did you build with the standard
 config?
 
 What is the standard config? As the operating system is Scientific
 Linux 6, I'm using the config for the standard 2.6.32 kernel with as
 little modifications as possible to use it for 3.3.2.

Sorry, meant something like standard distro config, i.e. not
localmodconfig or customised.

 I don't have a 4 core Xeon system, but I tried to simulate it on a
 dual Xeon 2670 system by only using 4 cores:
 
 [root@node037 src]# time numactl --physcpubind=0-3 make -j 4
 
 ...
 
 real11m45.748s
 user38m30.405s
 sys 4m48.634s

3.8.2 with standard debian config on 4 core xeon:

real28m19.009s
user84m16.092s
sys 7m18.327s

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Sven Barth

Am 06.03.2013 11:45, schrieb Florian Klaempfl:



Conclusion:

With FPC, the optimization of threadvar access is not necessary. Instead
it is highly recommended to always use thread specific instances of the
classes that do the threaded work. TThread of course provides this out
of the box.



And how does one the current TThread?

Since a few weeks: TThread.CurrentThread ;)

Regards,
Sven
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Michael Schnell

On 03/06/2013 01:43 PM, Sven Barth wrote:



And how does one the current TThread?

Since a few weeks: TThread.CurrentThread ;)



This does in fact use a threadvar:

threadvar
  { the instance of the current thread; in case of an external thread 
this is
Nil until TThread.GetCurrentThread was called once (the RTLs need 
to ensure

that threadvars are initialized with 0!) }
  CurrentThreadVar: TThread;


and (unless the compiler optimizes this out) even accesses it twice:


  if not Assigned(CurrentThreadVar) then
CurrentThreadVar := TExternalThread.Create;
  Result := CurrentThreadVar;


From within the TThread object, simply using Self seems more 
appropriate unless there is a chance that the same TThread instance is 
used for multiple OS-Threads. I don't know if/how this is possible.


From outside I feel that AnyThread.GetCurrentThread does not make 
much senses.


When doing TThread.GetCurrentThread as a class function I think I 
should get self of same when I am in the code that is called from 
Execute of some TThread instance. I  don't see what I want to see when 
I'm not.


The code seems to try to avoid the case that a no TThread instance when  
GetCurrentThread is called as a a class function. I don't know if/how 
this is sensible.


Using CurrentThread only seems sensible within the the code of a 
component that has been called by the code of a TThread instance. But 
here using an appropriate back-link property can easily be used to avoid 
accessing the threadvar. (or using  CurrentThread once to set a 
property  and then just accessing same.) But this of course needs to be 
done in user code and the RTL can't force it.


-Michael
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Sven Barth

Am 06.03.2013 14:29, schrieb Michael Schnell:

On 03/06/2013 01:43 PM, Sven Barth wrote:



And how does one the current TThread?

Since a few weeks: TThread.CurrentThread ;)



This does in fact use a threadvar:

threadvar
  { the instance of the current thread; in case of an external thread 
this is
Nil until TThread.GetCurrentThread was called once (the RTLs need 
to ensure

that threadvars are initialized with 0!) }
  CurrentThreadVar: TThread;


and (unless the compiler optimizes this out) even accesses it twice:


  if not Assigned(CurrentThreadVar) then
CurrentThreadVar := TExternalThread.Create;
  Result := CurrentThreadVar;


From within the TThread object, simply using Self seems more 
appropriate unless there is a chance that the same TThread instance is 
used for multiple OS-Threads. I don't know if/how this is possible.


From outside I feel that AnyThread.GetCurrentThread does not make 
much senses.


When doing TThread.GetCurrentThread as a class function I think I 
should get self of same when I am in the code that is called from 
Execute of some TThread instance. I  don't see what I want to see 
when I'm not.


The code seems to try to avoid the case that a no TThread instance 
when  GetCurrentThread is called as a a class function. I don't know 
if/how this is sensible.


Using CurrentThread only seems sensible within the the code of a 
component that has been called by the code of a TThread instance. But 
here using an appropriate back-link property can easily be used to 
avoid accessing the threadvar. (or using  CurrentThread once to set a 
property  and then just accessing same.) But this of course needs to 
be done in user code and the RTL can't force it.


The TThread.CurrentThread is mainly for access in functions that don't 
get passed a TThread instance and to also get a TThread instance for 
threads not created by the RTL (the TExternalThread.Create line 
above). Don't forget that TThread.CurrentThread is a class 
property/function, so you can't access Self.


We could optimize it like this:

=== code begin ===

Result := CurrentThreadVar;
if not Assigned(Result) then begin
  Result := TExternalThread.Create;
  CurrentThreadVar := Result;
end;

=== code end ===

So that in the normal case only one access to the threadvar is used.

Regards,
Sven
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Ivanko B
make -j has a dramatic effect on an SMP system, particularly if it can
find groups of jobs without too much interdependence.
=
The benchamark was surprising. Diring it, me observed 80..95% load of
each CPU still having high I/O load on the RAID1. How can it be ?!
Which were the CPUs engaged for ?
Me guess because of the design flaw of Core-I* arch where proper
(disk/network- to-from-RAM leaving CPUs idle from I/O) DMA transfers
are impossible since the RAM can only be accessed via CPUs so CPUs has
to serve each smallest I/O operation. Really very strange.
The kernel is customized, sure :)
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-06 Thread Ivanko B
time make -j 64
==
It might not build modules - which present a lot of files thsu I/O.
make -j .. deb-pkg builds  packs both image  modules (the
benchmark has selected for build approx 75% of all available modules )
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Mark Morgan Lloyd

Marco van de Voort wrote:


But even when in theory (which I btw don't even want to consider), you are
equivalent to C in this way, it basically means disabling the unit system,
and users must start to manual maintain dependencies, and learn to
interpretate cryptic errormessages if an incremental build goes haywire.

C users and developers are trained in this, and have their experience in
detangling the web of deps etc, have developed semi-automated helper tools
etc.

Inflicting this on the Pascal masses is unrealistic and undesirable.
Sticking to the manual build principles because the FPC devels can handle it
essentially means that nobody else will have parallel builds, or will resort
to a system of doing full builds only. (but that is throwing away the big
savings to gain small ones). Something that big C projects resort to anyway,
I'm told.

And FPC even only in a few critical points.

Manual maintenance is simply too painful (and atypical for modular languages and
its users).


But on the other hand, if an application programmer could disable FPC's 
unit handling and use  make -j  instead, choosing to pay the price of 
difficult maintenance, it might defuse the criticism coming from certain 
quarters.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Mark Morgan Lloyd

Sven Barth wrote:

Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:

Marco van de Voort wrote:

But even when in theory (which I btw don't even want to consider), 
you are
equivalent to C in this way, it basically means disabling the unit 
system,

and users must start to manual maintain dependencies, and learn to
interpretate cryptic errormessages if an incremental build goes haywire.

C users and developers are trained in this, and have their experience in
detangling the web of deps etc, have developed semi-automated helper 
tools

etc.

Inflicting this on the Pascal masses is unrealistic and undesirable.
Sticking to the manual build principles because the FPC devels can 
handle it
essentially means that nobody else will have parallel builds, or will 
resort
to a system of doing full builds only. (but that is throwing away the 
big
savings to gain small ones). Something that big C projects resort to 
anyway,

I'm told.

And FPC even only in a few critical points.

Manual maintenance is simply too painful (and atypical for modular 
languages and

its users).


But on the other hand, if an application programmer could disable 
FPC's unit handling and use  make -j  instead, choosing to pay the 
price of difficult maintenance, it might defuse the criticism coming 
from certain quarters.


Make can not figure out the dependencies between units by itself as it 
would need to parse them.


That's for the user to do, if he thinks he can do a better job than FPC.

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Sven Barth

Am 05.03.2013 10:41, schrieb Mark Morgan Lloyd:

Sven Barth wrote:

Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:

Marco van de Voort wrote:

But even when in theory (which I btw don't even want to consider), 
you are
equivalent to C in this way, it basically means disabling the unit 
system,

and users must start to manual maintain dependencies, and learn to
interpretate cryptic errormessages if an incremental build goes 
haywire.


C users and developers are trained in this, and have their 
experience in
detangling the web of deps etc, have developed semi-automated 
helper tools

etc.

Inflicting this on the Pascal masses is unrealistic and undesirable.
Sticking to the manual build principles because the FPC devels can 
handle it
essentially means that nobody else will have parallel builds, or 
will resort
to a system of doing full builds only. (but that is throwing away 
the big
savings to gain small ones). Something that big C projects resort 
to anyway,

I'm told.

And FPC even only in a few critical points.

Manual maintenance is simply too painful (and atypical for modular 
languages and

its users).


But on the other hand, if an application programmer could disable 
FPC's unit handling and use  make -j  instead, choosing to pay the 
price of difficult maintenance, it might defuse the criticism coming 
from certain quarters.


Make can not figure out the dependencies between units by itself as 
it would need to parse them.


That's for the user to do, if he thinks he can do a better job than FPC.

In theory you can do it already. Compile each unit with -Ur (release 
unit) and the compiler will not attempt to recompile such a unit and 
prefer to fail. And then you do a compilation run for each unit. Good 
luck though for circular dependencies (the legal ones).


Regards,
Sven
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Henry Vermaak
On Tue, Mar 05, 2013 at 09:41:37AM +, Mark Morgan Lloyd wrote:
 Sven Barth wrote:
 Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:
 
 But on the other hand, if an application programmer could
 disable FPC's unit handling and use  make -j  instead, choosing
 to pay the price of difficult maintenance, it might defuse the
 criticism coming from certain quarters.
 
 Make can not figure out the dependencies between units by itself
 as it would need to parse them.
 
 That's for the user to do, if he thinks he can do a better job than FPC.

Or implement an option that spits out the dependencies for make, so that
the user doesn't have to do this.

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Sven Barth

Am 05.03.2013 10:58, schrieb Henry Vermaak:

On Tue, Mar 05, 2013 at 09:41:37AM +, Mark Morgan Lloyd wrote:

Sven Barth wrote:

Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:

But on the other hand, if an application programmer could
disable FPC's unit handling and use  make -j  instead, choosing
to pay the price of difficult maintenance, it might defuse the
criticism coming from certain quarters.


Make can not figure out the dependencies between units by itself
as it would need to parse them.

That's for the user to do, if he thinks he can do a better job than FPC.

Or implement an option that spits out the dependencies for make, so that
the user doesn't have to do this.
As this is something we (as in the FPC developers) don't need: patches 
are welcome.


[or you could try whether the ppdep tool still works: 
http://www.freepascal.org/tools/ppdep.var ]


Regards,
Sven
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Mark Morgan Lloyd

Henry Vermaak wrote:

On Tue, Mar 05, 2013 at 09:41:37AM +, Mark Morgan Lloyd wrote:

Sven Barth wrote:

Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:

But on the other hand, if an application programmer could
disable FPC's unit handling and use  make -j  instead, choosing
to pay the price of difficult maintenance, it might defuse the
criticism coming from certain quarters.


Make can not figure out the dependencies between units by itself
as it would need to parse them.

That's for the user to do, if he thinks he can do a better job than FPC.


Or implement an option that spits out the dependencies for make, so that
the user doesn't have to do this.


Possibly augmented by timestamp-based validity checking. Yes, but I 
didn't want to muddy the water by suggesting too many things at once :-)


Seriously: if FPC's putting too much internal work into sorting out 
dependencies and somebody is prepared to write his code for an 
alternative build system, then give him enough rope. But it would 
obviously shine a harsh light on the compiler if this turned out not to 
be the major bottleneck.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Michael Van Canneyt



On Tue, 5 Mar 2013, Sven Barth wrote:


Am 05.03.2013 10:58, schrieb Henry Vermaak:

On Tue, Mar 05, 2013 at 09:41:37AM +, Mark Morgan Lloyd wrote:

Sven Barth wrote:

Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:

But on the other hand, if an application programmer could
disable FPC's unit handling and use  make -j  instead, choosing
to pay the price of difficult maintenance, it might defuse the
criticism coming from certain quarters.


Make can not figure out the dependencies between units by itself
as it would need to parse them.

That's for the user to do, if he thinks he can do a better job than FPC.

Or implement an option that spits out the dependencies for make, so that
the user doesn't have to do this.
As this is something we (as in the FPC developers) don't need: patches are 
welcome.


[or you could try whether the ppdep tool still works: 
http://www.freepascal.org/tools/ppdep.var ]


There is a new tool pas2fpm.pp which can easily be adapted to do this.
It already calculates the dependencies, but outputs them in fpmake form.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Henry Vermaak
On Tue, Mar 05, 2013 at 11:05:22AM +0100, Sven Barth wrote:
 Am 05.03.2013 10:58, schrieb Henry Vermaak:
 On Tue, Mar 05, 2013 at 09:41:37AM +, Mark Morgan Lloyd wrote:
 Sven Barth wrote:
 Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:
 But on the other hand, if an application programmer could
 disable FPC's unit handling and use  make -j  instead, choosing
 to pay the price of difficult maintenance, it might defuse the
 criticism coming from certain quarters.
 
 Make can not figure out the dependencies between units by itself
 as it would need to parse them.
 That's for the user to do, if he thinks he can do a better job than FPC.
 Or implement an option that spits out the dependencies for make, so that
 the user doesn't have to do this.
 As this is something we (as in the FPC developers) don't need:
 patches are welcome.

I'm trying to ascertain if this is even possible (the c-style,
file-at-a-time compilation, using make to handle multiple processes).
Do you think it's possible, then?

I'm not ordering anyone to implement features, I'm exploring solutions
to the depressing fact that my c projects compile quicker than my fpc
projects (simply because my build system can launch parallel jobs).  I
don't think that turning fpc into a multi-threaded compiler is a sane
thing to do.

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Mattias Gaertner

Michael Van Canneyt mich...@freepascal.org hat am 5. März 2013 um 11:09
geschrieben:


 On Tue, 5 Mar 2013, Sven Barth wrote:

  Am 05.03.2013 10:58, schrieb Henry Vermaak:
  On Tue, Mar 05, 2013 at 09:41:37AM +, Mark Morgan Lloyd wrote:
  Sven Barth wrote:
  Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:
  But on the other hand, if an application programmer could
  disable FPC's unit handling and use make -j instead, choosing
  to pay the price of difficult maintenance, it might defuse the
  criticism coming from certain quarters.
 
  Make can not figure out the dependencies between units by itself
  as it would need to parse them.
  That's for the user to do, if he thinks he can do a better job than FPC.
  Or implement an option that spits out the dependencies for make, so that
  the user doesn't have to do this.
  As this is something we (as in the FPC developers) don't need: patches are
  welcome.
 
  [or you could try whether the ppdep tool still works:
  http://www.freepascal.org/tools/ppdep.var ]

 There is a new tool pas2fpm.pp which can easily be adapted to do this.
 It already calculates the dependencies, but outputs them in fpmake form.

Is this correct:
You need the tool pas2fpm to feed the tool fpmake to feed the tool fpc to feed
linker, lazres 


Mattias
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Henry Vermaak
On Tue, Mar 05, 2013 at 11:09:39AM +0100, Michael Van Canneyt wrote:
 
 There is a new tool pas2fpm.pp which can easily be adapted to do this.
 It already calculates the dependencies, but outputs them in fpmake form.

Ah, I remember that fpmake can build with multiple threads, so perhaps
this is a better solution than to do it with make.  I'll investigate.

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Michael Van Canneyt



On Tue, 5 Mar 2013, Mattias Gaertner wrote:



Michael Van Canneyt mich...@freepascal.org hat am 5. März 2013 um 11:09
geschrieben:



On Tue, 5 Mar 2013, Sven Barth wrote:


Am 05.03.2013 10:58, schrieb Henry Vermaak:

On Tue, Mar 05, 2013 at 09:41:37AM +, Mark Morgan Lloyd wrote:

Sven Barth wrote:

Am 05.03.2013 10:14, schrieb Mark Morgan Lloyd:

But on the other hand, if an application programmer could
disable FPC's unit handling and use make -j instead, choosing
to pay the price of difficult maintenance, it might defuse the
criticism coming from certain quarters.


Make can not figure out the dependencies between units by itself
as it would need to parse them.

That's for the user to do, if he thinks he can do a better job than FPC.

Or implement an option that spits out the dependencies for make, so that
the user doesn't have to do this.

As this is something we (as in the FPC developers) don't need: patches are
welcome.

[or you could try whether the ppdep tool still works:
http://www.freepascal.org/tools/ppdep.var ]


There is a new tool pas2fpm.pp which can easily be adapted to do this.
It already calculates the dependencies, but outputs them in fpmake form.


Is this correct:
You need the tool pas2fpm to feed the tool fpmake to feed the tool fpc to feed
linker, lazres 


You don't need anything. You can create fpmake.pp yourself.

pas2fpm makes it easier for you.

Just like configure and autoconf make it easier to create Makefiles if you 
use C.

But because pas2fpm parses your units to do its job, it can go wrong, even on 
legal code.
So using it should be optional.

You could try and integrate all in 1 tool (like in an IDE as lazarus).
fpmake was set up so it can be integrated completely in lazarus if you want it. 
Same for fppkg.


Michael.___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Marco van de Voort
In our previous episode, Henry Vermaak said:
  IMHO a threading clusterfsck is preferable to a forking clusterfck :-)
 
 My gut feeling would be that the complexity and potential bugs/races
 don't make up for the speed, but maybe a threaded compiler gains a lot
 more than I imagine.  Are there any multi-threaded compilers around for
 comparison?

Not the normal free ones. But that is not really a metric, but none of them 
were designed for modular
languages, stronger, they don't even use features like precompiled headers
that commercial ones already have for a long time.
 
Note that strictly only the part of make *must* be multithreaded, the
compilation of a compilation unit is basically the current single threaded
compiler running in its own thread.

So the parts that decide which compilation unit must be loaded, and parts of
the unit loader.

I think the problem is more doing it in a non-trivial production compiler, 
than the principle itself.

  This means you need to manually specify in the build system which
  compilation unit depends on which headers. And this is also why in C/C++
  inline functions are in the header, so that this model is persistent, since
  most builds are not full, but incremental.
 
 As you probably know, you can use -MD and variants with gcc, which
 outputs dependency files for make, that you then include in the
 Makefile. 

No, but I did know other tools for that yes. Compiler output is better of
course.

 If you add, e.g. -MMD to the CFLAGS, these dependencies are
 generated as a side effect of the compilation process, so it's not a
 separate step.  As a result, partial builds work really well, and can be
 parallelized as usual.  No manual work involved.

Maybe that works for FPC too. But FPC can currently compile units multiple
times afaik.  (to detangle circular references), FPC afaik also registers
some preprocessor state, and compiles on change.

So basically make is meant for a different execution model IMHO, and it
shouldn't be made a cornerstone. It is only a stopgap till we have something
that fits with the unit system.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Marco van de Voort
In our previous episode, Henry Vermaak said:
  But on the other hand, if an application programmer could
  disable FPC's unit handling and use  make -j  instead, choosing
  to pay the price of difficult maintenance, it might defuse the
  criticism coming from certain quarters.
  
  Make can not figure out the dependencies between units by itself
  as it would need to parse them.
  
  That's for the user to do, if he thinks he can do a better job than FPC.
 
 Or implement an option that spits out the dependencies for make, so that
 the user doesn't have to do this.

That does't work, since in FPC an unit is an interface AND an
implementation. And inline functions are in the implementation, not the 
interface.

The only way is to simply toss the modular system (units) out, and work with
$i, declare functions as externals in headers, let the compiler spit out
dependencies.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Graeme Geldenhuys
On 2013-03-05 10:31, Henry Vermaak wrote:
 
 Ah, I remember that fpmake can build with multiple threads, so perhaps
 this is a better solution than to do it with make.  I'll investigate.

I've got that option enabled by default for building FPC 2.7.1 and it
does shave off a few seconds.

I'm also trying to use fpmake for other hourly build server tasks,
where I need to do clean compiles of various independent packages first,
then build the test suite that pulls everything together. eg: building
Synapse, FBLib, tiOPF, EpikTimer, fpGUI etc in parallel. Then somehow
wait until they are all done, then build the test suite which uses all
those packages. I still haven't figure this process out yet. If you have
hints or suggestion on this, it would be greatly appreciated [maybe
follow-up in a new message thread in fpc-pascal]


Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Henry Vermaak
On Tue, Mar 05, 2013 at 11:44:37AM +0100, Marco van de Voort wrote:
 In our previous episode, Henry Vermaak said:
   But on the other hand, if an application programmer could
   disable FPC's unit handling and use  make -j  instead, choosing
   to pay the price of difficult maintenance, it might defuse the
   criticism coming from certain quarters.
   
   Make can not figure out the dependencies between units by itself
   as it would need to parse them.
   
   That's for the user to do, if he thinks he can do a better job than FPC.
  
  Or implement an option that spits out the dependencies for make, so that
  the user doesn't have to do this.
 
 That does't work, since in FPC an unit is an interface AND an
 implementation. And inline functions are in the implementation, not the 
 interface.
 
 The only way is to simply toss the modular system (units) out, and work with
 $i, declare functions as externals in headers, let the compiler spit out
 dependencies.

OK, I'll drop this line of enquiry then :)  Thanks for the information
(and the patience).

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Hans-Peter Diettrich

Henry Vermaak schrieb:


I'm trying to ascertain if this is even possible (the c-style,
file-at-a-time compilation, using make to handle multiple processes).
Do you think it's possible, then?


Yes and no. It would mean to create kind of header files for the Pascal 
units, usable to compile the units independently. No problem at a first 
glance, the Interface is the equivalent of an C header file, and the 
Uses lists are equivalent to further #includes. One of the remaining 
problems are the lost unit qualifiers, so that external references in 
the object files cannot be safely resolved.


But in fact above is what FPC already does! When it encounters a Uses 
list, it loads the according PPU or, if not available or outdated, 
compiles the requested unit. Thus the PPU files already *are* the 
equivalent of the C header files. Plus the bonus that every header has 
to be loaded only once, not for every single module to be compiled by an 
C compiler.


The problem IMO is the long list of units waiting for their dependencies 
loaded, before the first unit can be compiled at all (top-down). More 
efficient were bottom-up compilation, starting with the units with only 
few dependencies, so that compilation could start while the disk is 
stressed by loading all other PPU files, required for the compilation of 
units with more dependencies.


DoDi

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread waldo kitty

On 3/5/2013 05:50, Graeme Geldenhuys wrote:

I'm also trying to use fpmake for other hourly build server tasks,
where I need to do clean compiles of various independent packages first,
then build the test suite that pulls everything together. eg: building
Synapse, FBLib, tiOPF, EpikTimer, fpGUI etc in parallel. Then somehow
wait until they are all done, then build the test suite which uses all
those packages. I still haven't figure this process out yet. If you have
hints or suggestion on this, it would be greatly appreciated [maybe
follow-up in a new message thread in fpc-pascal]


on another system i work with, we use disk-based breadcrumb semaphore files for 
the different stages and parts of those stages... one is created when the 
section is entered into and another when the section is successfully compiled... 
it is mostly all C stuff and uses make... it is all pretty much FM to me as i 
only start it and wait for it to reach my modules and see if they compile or 
fail... if they fail, i (w)hack at them some more... the breadcrumbs allow the 
compilation environment to pick up where it left off instead of spending two 
hours recompiling everything again up to the point of failure...


maybe this can give you some ideas?

FWIW: the above breadcrumb system that i speak of is in the development version 
of an open source firewall product that i work with ;)

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Ivanko B
so that compilation could start while the disk is stressed by loading
all other PPU files, required for the compilation of units with more
dependencies.
==
Disk I/O is a huge low-down to avoid on any price (like databases do
with their indexing). Today me tested building LINUX kernel 3.8.0 on
8*SMT=16 CPU system (Corei7 2G) + 32G DDR3 + 175(wr)/350(rd)MB/s
RAID1. And this monster outperformed another cheap E8400 + 4G machine
by only 2 times and most probably because the cheap machine was
non-RAIDed.  And because Core i* arch can't have proper DMA.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Mark Morgan Lloyd

Ivanko B wrote:

so that compilation could start while the disk is stressed by loading
all other PPU files, required for the compilation of units with more
dependencies.
==
Disk I/O is a huge low-down to avoid on any price (like databases do
with their indexing). Today me tested building LINUX kernel 3.8.0 on
8*SMT=16 CPU system (Corei7 2G) + 32G DDR3 + 175(wr)/350(rd)MB/s
RAID1. And this monster outperformed another cheap E8400 + 4G machine
by only 2 times and most probably because the cheap machine was
non-RAIDed.  And because Core i* arch can't have proper DMA.


I've not had an opportunity to try this, but my understanding is that on 
a Sun E10K with something like 256 400MHz processors the Linux kernel 
built in around 20 seconds. I've had it build in about 3 minutes on a 
system with 16x 80MHz processors, but that was in the days of kernel 2.2 
and there was probably less than half today's number of files involved.


make -j has a dramatic effect on an SMP system, particularly if it can 
find groups of jobs without too much interdependence. If there's a lot 
of shared input files etc., then my experience is that it tends to level 
out at around -j 8 since it's extremely difficult to improve the cache 
architecture beyond that point.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Daniël Mantione



Op Tue, 5 Mar 2013, schreef Mark Morgan Lloyd:

I've not had an opportunity to try this, but my understanding is that on a 
Sun E10K with something like 256 400MHz processors the Linux kernel built in 
around 20 seconds. I've had it build in about 3 minutes on a system with 16x 
80MHz processors, but that was in the days of kernel 2.2 and there was 
probably less than half today's number of files involved.


Well... I put that into question. I don't have a 256 core system, but I 
have a quad Opteron 6376 system, 64 cores @ 2.4 GHz is quite a bit of 
compute power. Compiling kernel 3.3.2:


[root@node016 linux-3.3.2]# time make -j 64

...

real1m54.823s
user77m14.178s
sys 11m32.109s

To minutes to build a kernel is still fast though :)

Daniël___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Graeme Geldenhuys
On 2013-03-05 17:09, waldo kitty wrote:
 
 on another system i work with, we use disk-based breadcrumb semaphore files 
 for 
 the different stages and parts of those stages...

Thanks for you input. It sounds similar to what I was planning. Simply
create an empty file in /tmp when each independent package is compiled
successfully. Another process keeps checking /tmp for all the 0 byte
files in expects. If they all exist, then build the test suite. Once
that is complete, remove all the 0 byte files, and run the test suite.

Now the question is, how good is fpmake? Will it can do something like
that, or must I build my own system using scripts or console apps etc.


Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Sven Barth

On 05.03.2013 20:58, Graeme Geldenhuys wrote:

On 2013-03-05 17:09, waldo kitty wrote:


on another system i work with, we use disk-based breadcrumb semaphore files for
the different stages and parts of those stages...


Thanks for you input. It sounds similar to what I was planning. Simply
create an empty file in /tmp when each independent package is compiled
successfully. Another process keeps checking /tmp for all the 0 byte
files in expects. If they all exist, then build the test suite. Once
that is complete, remove all the 0 byte files, and run the test suite.

Now the question is, how good is fpmake? Will it can do something like
that, or must I build my own system using scripts or console apps etc.


I didn't yet experiment that much with fpmake, but considering that 
fpmake.pas files are just plain Pascal programs...


Regards,
Sven

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Henry Vermaak
On Tue, Mar 05, 2013 at 07:26:19PM +0100, Daniël Mantione wrote:
 
 
 Op Tue, 5 Mar 2013, schreef Mark Morgan Lloyd:
 
 I've not had an opportunity to try this, but my understanding is
 that on a Sun E10K with something like 256 400MHz processors the
 Linux kernel built in around 20 seconds. I've had it build in
 about 3 minutes on a system with 16x 80MHz processors, but that
 was in the days of kernel 2.2 and there was probably less than
 half today's number of files involved.
 
 Well... I put that into question. I don't have a 256 core system,
 but I have a quad Opteron 6376 system, 64 cores @ 2.4 GHz is quite a
 bit of compute power. Compiling kernel 3.3.2:
 
 [root@node016 linux-3.3.2]# time make -j 64
 
 ...
 
 real1m54.823s
 user77m14.178s
 sys 11m32.109s

Damn.  My custom config kernel compiles stable kernels in 3-5 minutes on
a quad core Xeon, which isn't bad.  Did you build with the standard
config?

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-05 Thread Mark Morgan Lloyd

Henry Vermaak wrote:

On Tue, Mar 05, 2013 at 07:26:19PM +0100, Daniël Mantione wrote:


Op Tue, 5 Mar 2013, schreef Mark Morgan Lloyd:


I've not had an opportunity to try this, but my understanding is
that on a Sun E10K with something like 256 400MHz processors the
Linux kernel built in around 20 seconds. I've had it build in
about 3 minutes on a system with 16x 80MHz processors, but that
was in the days of kernel 2.2 and there was probably less than
half today's number of files involved.

Well... I put that into question. I don't have a 256 core system,
but I have a quad Opteron 6376 system, 64 cores @ 2.4 GHz is quite a
bit of compute power. Compiling kernel 3.3.2:

[root@node016 linux-3.3.2]# time make -j 64

...

real1m54.823s
user77m14.178s
sys 11m32.109s


Damn.  My custom config kernel compiles stable kernels in 3-5 minutes on
a quad core Xeon, which isn't bad.  Did you build with the standard
config?


Noting that for this test to be valid you obviously have to start from 
e.g.  make clean  and might usefully consider a reboot to make sure that 
all caches are empty.


My point however was that a kernel build parallelises to very good 
effect, even where each processor is implausibly slow by today's 
standards.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Michael Schnell

On 03/02/2013 04:02 PM, Florian Klaempfl wrote:


In theory yes but I still fear that the threadvars and synchronization 
eats much of the advantage in this case.


I suppose the (high level) synchronization will be rather complex and 
eat performance.


The low level threadvar implementation is known to be improvable. AFAIR, 
it uses library calls as well in Windows as in Linux while the 
appropriate C  compilers use segment registers (on x86 / x86-64) which 
are documented in Linux and undocumented in Windows and supposedly a lot 
faster.


OTOH, are Theradvars necessary to be used very frequently (i.e. more 
than when creating thread-specific objects, that then can be used to 
access the thread specific data) ?


-Michael
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Michael Van Canneyt



On Mon, 4 Mar 2013, Michael Schnell wrote:


On 03/02/2013 04:02 PM, Florian Klaempfl wrote:


In theory yes but I still fear that the threadvars and synchronization eats 
much of the advantage in this case.


I suppose the (high level) synchronization will be rather complex and eat 
performance.


The low level threadvar implementation is known to be improvable. AFAIR, it 
uses library calls as well in Windows as in Linux while the appropriate C 
compilers use segment registers (on x86 / x86-64) which are documented in 
Linux and undocumented in Windows and supposedly a lot faster.


The (on x86 / x86-64) disqualifies your argument for use. 
The solution must work on ALL platforms...


BTW. threadvars in FPC are done in the same way as in Delphi, with TLS.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Michael Schnell

On 03/04/2013 09:49 AM, Michael Van Canneyt wrote:

 The solution must work on ALL platforms...


Of course.

I don't remember ever having seen a system (Windows, Linux, x86, ARM, 
NIOS (similar top MIPS) ) where the C compiler does a library call when 
accessing a threadvar. Usually a register is dedicated for this purpose 
that is loaded with a thread specific value by the OS. I do know C 
compiler implementations that don't support threadvars at all and thus 
are unable to support FUTEX and with that low level thread 
synchronization is a lot slower as always a userland/system switch is 
necessary. Here hardware (atomic instructions) and the OS implementation 
(e.g. Atomic Region to overcome such limitations for userland) comes 
into play.


I remember from a discussion some years ago that fpc uses a (supposedly 
arch/platform depending) library call with any access to to a threadvar 
instead of using the appropriate (arch/platform depending) register. But 
I did not recheck recently.


-Michael
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Michael Van Canneyt



On Mon, 4 Mar 2013, Michael Schnell wrote:


On 03/04/2013 09:49 AM, Michael Van Canneyt wrote:

 The solution must work on ALL platforms...


Of course.

I don't remember ever having seen a system (Windows, Linux, x86, ARM, NIOS 
(similar top MIPS) ) where the C compiler does a library call when accessing 
a threadvar. Usually a register is dedicated for this purpose that is loaded 
with a thread specific value by the OS. I do know C compiler implementations 
that don't support threadvars at all and thus are unable to support FUTEX and 
with that low level thread synchronization is a lot slower as always a 
userland/system switch is necessary. Here hardware (atomic instructions) and 
the OS implementation (e.g. Atomic Region to overcome such limitations for 
userland) comes into play.


I remember from a discussion some years ago that fpc uses a (supposedly 
arch/platform depending) library call with any access to to a threadvar 
instead of using the appropriate (arch/platform depending) register. But I 
did not recheck recently.


We use pthreads on non-windows, and GetTLS/SetTLS on Windows.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Michael Schnell

On 03/04/2013 10:07 AM, Michael Van Canneyt wrote:


We use pthreads on non-windows, and GetTLS/SetTLS on Windows.

Thus the situation is as it was when we discussed this some years ago: a 
library call with each access to a threadvar, making fpc a lot slower 
that the C compiler (Microsoft and GNU, no idea about other brands) on 
that behalf (that might nob be very relevant in real life) .


(BTW: a similar discussion arises with any thread-synchronization 
primitive AFAIK here dynamic library calls are done while an 
arch-depending Futex/Mutex implementation directly in the RTL _could_ be 
done. But here, a C user (Linux) would do the same library call to 
pthreadlib mutex...() which does Futexif available in that arch, and 
Mutex if not. )


-Michael
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Henry Vermaak
On Sat, Mar 02, 2013 at 05:26:02PM +0100, Marco van de Voort wrote:
 In our previous episode, Henry Vermaak said:
   I don't see why there would be more synchronization overhead than make?
  
  So why not keep it simple and let the build system handle parallel jobs?
 
 I like autobuilding. IMHO that is a core attraction of unit based (modular)
 languages.
 
 Manually maintaining dependencies between compilation units is stone-age.

I just add all the objects to a variable in a Makefile.

The result is that I have a 27000 line c library that compiles in *half*
the time it takes to compile a 4000 line lazarus program because my c
lib can be built with make -j 9.

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Marco van de Voort
In our previous episode, Henry Vermaak said:
  Manually maintaining dependencies between compilation units is stone-age.
 
 I just add all the objects to a variable in a Makefile.
 
 The result is that I have a 27000 line c library that compiles in *half*
 the time it takes to compile a 4000 line lazarus program because my c
 lib can be built with make -j 9.

I'm not exactly sure what this proves. The C situation is not equivalent to
the Pascal situation, the latter trying to make sure all (Pascal level
symbols) are there before the linker.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Tomas Hajny
On Mon, March 4, 2013 10:33, Michael Schnell wrote:
 On 03/04/2013 10:07 AM, Michael Van Canneyt wrote:

 We use pthreads on non-windows, and GetTLS/SetTLS on Windows.

 Thus the situation is as it was when we discussed this some years ago: a
 library call with each access to a threadvar, making fpc a lot slower
 that the C compiler (Microsoft and GNU, no idea about other brands) on
 that behalf (that might nob be very relevant in real life) .
 .
 .

It's questionable whether it should be responsibility of a compiler to
provide efficient support for threadvars, or whether the target platform
should provide more efficient support for them in its design and its
libraries (here MS Windows API, Posix threads, etc.). The OS/2 target does
not need any API call for accessing threadvars (thanks to the OS/2 design
using virtualization features provided by the Intel architecture) and
there is no compiler magic involved. Admittedly, this may not scale well
to all types of CPU architectures (i.e. function call to a platform API
may be more appropriate in the end to allow simpler and more powerful CPU
architectures alike), but there is no need why the platform API should be
more complicated (i.e. slower) than the simple pointer access (as used
with the OS/2 implementation for threadvar access) on a CPU architecture
providing such features.

Tomas


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Michael Schnell

On 03/04/2013 12:29 PM, Tomas Hajny wrote:
It's questionable whether it should be responsibility of a compiler to 
provide efficient support for threadvars, 


If the target platform does not support threadvars a compiler can't do 
it at all.


If the target platform only provides library calls, a compiler needs to 
use them.


If the target platform provides dedicated pointer registers for 
threadvars a compiler can use them.


In all fpc enabled patforms I examined (X86 Windows, X86 Linux, X64-64 
Windows, X86-64 Linux, ARM Linux (32 Bits) the platform provides as well 
library calls as dedicated threadvar pointer registers. In windows the 
registers seem to be undocumented but obviously stable, as a change 
would break a lot existing C/C++ software.


All C/C++ compilers I examined used the threadvar pointer Registers for 
speed..


-Michael
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Henry Vermaak
On Mon, Mar 04, 2013 at 11:19:38AM +0100, Marco van de Voort wrote:
 In our previous episode, Henry Vermaak said:
   Manually maintaining dependencies between compilation units is stone-age.
  
  I just add all the objects to a variable in a Makefile.
  
  The result is that I have a 27000 line c library that compiles in *half*
  the time it takes to compile a 4000 line lazarus program because my c
  lib can be built with make -j 9.
 
 I'm not exactly sure what this proves. The C situation is not equivalent to
 the Pascal situation, the latter trying to make sure all (Pascal level
 symbols) are there before the linker.

What I'm trying to say is that gcc is going 3 times faster on my quad
core machine because of the build system, not because they've turned
their compiler into a multi-threaded clusterfsck.  In my ignorance, I'm
assuming that the same can't be achieved with pascal, then (without
making the compiler itself multi-threaded)?

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Marco van de Voort
In our previous episode, Henry Vermaak said:
   I just add all the objects to a variable in a Makefile.
   
   The result is that I have a 27000 line c library that compiles in *half*
   the time it takes to compile a 4000 line lazarus program because my c
   lib can be built with make -j 9.
  
  I'm not exactly sure what this proves. The C situation is not equivalent to
  the Pascal situation, the latter trying to make sure all (Pascal level
  symbols) are there before the linker.
 
 What I'm trying to say is that gcc is going 3 times faster on my quad
 core machine because of the build system, not because they've turned
 their compiler into a multi-threaded clusterfsck.

IMHO a threading clusterfsck is preferable to a forking clusterfck :-)

Yes. But is that incremental? It is fun to speed up a full build, but in
reality developers are more likely to do a partial one. Ever pressed F9
after a small change in Delphi? The app starts nearly immediately, if
the change wasn't too deep in the hierarchy of dependencies.

But compilation units in C have weaker(no) coherence than in (modular) Pascal, 
and C does
not have an auto build system. All dependencies are done manually.
(sometimes assisted with semi automated tools)

This means you need to manually specify in the build system which
compilation unit depends on which headers. And this is also why in C/C++
inline functions are in the header, so that this model is persistent, since
most builds are not full, but incremental.

To build (possibly incremental) an interweaved codebase means handling
complete control to make, and essentially means castrating FPC to the gcc
model of one process, one compilation unit (and killing all auto dependency
tracking and no recompile unless updated).

This also means that for every codebase dependencies must be maintained. 
There might be semi-automated tools for it, but it is always a manual
responsibility in the end.  This is more or less done for the RTL.  If it
fails however, the result is partial builds, and errors pascal people are
not used to.

Of course you can also try to identify isolated groups of uses that can
be build in parallel, and try to build those, with autobuild within
that group. This happens in the packages/ directory, where only the
dependency graph of whole dirs is maintained.

 assuming that the same can't be achieved with pascal, then (without
 making the compiler itself multi-threaded)?

Basically, there is a small piece of make in the compiler for incremental
building, and determining what sources need to be compiled (just handing
a mainmodule +dirs to the compiler will build it all).

This means the compiler can make decisions outside make's control, and also
does extra checks (to find a consistent state for the code that abides the
modular system before linking)

As shown above, you can work around that with makefile generators
(packages/) or hard work (rtl/). But  for dynamic codebases of closely knit
units like IDEs and compilers there really is no solution atm.

If you have an option to force the compiler to only compile one unit, you
could emulate the C model.  You could even improve on the C situation by
letting the compiler terminate on inconsistent incremental build state (to
signal the dependencies are fscked)

But even when in theory (which I btw don't even want to consider), you are
equivalent to C in this way, it basically means disabling the unit system,
and users must start to manual maintain dependencies, and learn to
interpretate cryptic errormessages if an incremental build goes haywire.

C users and developers are trained in this, and have their experience in
detangling the web of deps etc, have developed semi-automated helper tools
etc.

Inflicting this on the Pascal masses is unrealistic and undesirable.
Sticking to the manual build principles because the FPC devels can handle it
essentially means that nobody else will have parallel builds, or will resort
to a system of doing full builds only. (but that is throwing away the big
savings to gain small ones). Something that big C projects resort to anyway,
I'm told.

And FPC even only in a few critical points.

Manual maintenance is simply too painful (and atypical for modular languages and
its users).
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Henry Vermaak
On Mon, Mar 04, 2013 at 09:25:02PM +0100, Marco van de Voort wrote:
 In our previous episode, Henry Vermaak said:
I just add all the objects to a variable in a Makefile.

The result is that I have a 27000 line c library that compiles in *half*
the time it takes to compile a 4000 line lazarus program because my c
lib can be built with make -j 9.
   
   I'm not exactly sure what this proves. The C situation is not equivalent 
   to
   the Pascal situation, the latter trying to make sure all (Pascal level
   symbols) are there before the linker.
  
  What I'm trying to say is that gcc is going 3 times faster on my quad
  core machine because of the build system, not because they've turned
  their compiler into a multi-threaded clusterfsck.
 
 IMHO a threading clusterfsck is preferable to a forking clusterfck :-)

My gut feeling would be that the complexity and potential bugs/races
don't make up for the speed, but maybe a threaded compiler gains a lot
more than I imagine.  Are there any multi-threaded compilers around for
comparison?

 Yes. But is that incremental? It is fun to speed up a full build, but in
 reality developers are more likely to do a partial one. Ever pressed F9
 after a small change in Delphi? The app starts nearly immediately, if
 the change wasn't too deep in the hierarchy of dependencies.
 
 But compilation units in C have weaker(no) coherence than in (modular) 
 Pascal, and C does
 not have an auto build system. All dependencies are done manually.
 (sometimes assisted with semi automated tools)
 
 This means you need to manually specify in the build system which
 compilation unit depends on which headers. And this is also why in C/C++
 inline functions are in the header, so that this model is persistent, since
 most builds are not full, but incremental.

As you probably know, you can use -MD and variants with gcc, which
outputs dependency files for make, that you then include in the
Makefile.  If you add, e.g. -MMD to the CFLAGS, these dependencies are
generated as a side effect of the compilation process, so it's not a
separate step.  As a result, partial builds work really well, and can be
parallelized as usual.  No manual work involved.

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-04 Thread Ivanko B
C users and developers are trained in this, and have their experience in
detangling the web of deps etc, have developed semi-automated helper tools
etc.
---
Inflicting this on the Pascal masses is unrealistic and undesirable.
==
100% !
C(++) building system is a nightmare. It hardly worths to take.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Daniël Mantione



Op Sun, 3 Mar 2013, schreef Martin Schreiber:


BTW, a significant percentage of the time is waiting for FPC compiling because
FPC normally crashes without -B.


I think you should focus your efforts on getting those bugs fixed, such as 
by submitting bug reports that allow reproduction of the problem with 
programs as simple as you can get them to show the problem.


It is completely unrealistic that FPC will get the same compiler speed as 
Delphi. FPC will become slower in the future for sure, because the 
compiler will execute more algorithms to generate better code.


Daniël___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Martin Schreiber
On Sunday 03 March 2013 10:04:54 Daniël Mantione wrote:
 Op Sun, 3 Mar 2013, schreef Martin Schreiber:
  BTW, a significant percentage of the time is waiting for FPC compiling
  because FPC normally crashes without -B.

 I think you should focus your efforts on getting those bugs fixed, such as
 by submitting bug reports that allow reproduction of the problem with
 programs as simple as you can get them to show the problem.

The problem is well known since a long time AFAIK. The answer we normally get 
when we report a unit handling related FPC crash is that FPC unit handling 
should be rewritten. I can't rewrite FPC unit handling myself because I have 
many other tasks. ;-)

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Florian Klämpfl
Am 03.03.2013 10:46, schrieb Martin Schreiber:
 On Sunday 03 March 2013 10:04:54 Daniël Mantione wrote:
 Op Sun, 3 Mar 2013, schreef Martin Schreiber:
 BTW, a significant percentage of the time is waiting for FPC compiling
 because FPC normally crashes without -B.

 I think you should focus your efforts on getting those bugs fixed, such as
 by submitting bug reports that allow reproduction of the problem with
 programs as simple as you can get them to show the problem.

 The problem is well known since a long time AFAIK. The answer we normally get 
 when we report a unit handling related FPC crash is that FPC unit handling 
 should be rewritten. I can't rewrite FPC unit handling myself because I have 
 many other tasks. ;-)

Same here :)

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Sven Barth

On 03.03.2013 08:23, Martin Schreiber wrote:

On Sunday 03 March 2013 08:12:28 Martin Schreiber wrote:

On Saturday 02 March 2013 11:28:25 Michael Van Canneyt wrote:

On Sat, 2 Mar 2013, Martin Schreiber wrote:

On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:

Anyway, what seems obvious from your numbers is that it is the linking
stage that needs speedup. This does not really come as a surprise.


On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll
check it later, currently I try to make MSEide+MSEgui Kylix3 compatible
so we have a benchmark on Linux too.


When you do, please send me the output of a strace -f


I will take a while because it is much work to make the code both FPC and
Delphi/Kylix compatible.


BTW, a significant percentage of the time is waiting for FPC compiling because
FPC normally crashes without -B.


Do you mean compiling the compiler? If so than for real tests you SHOULD 
NOT compile the compiler by hand, but only through make cycle in the 
compiler directory or make all in the top level directory. This will 
ensure that the compiler is compiled correctly.


And yes, I know the problem with -B and it's likely related to unit 
loading. This normally happens only in more complex projects like the 
compiler. Though I've just managed yesterday to create a smaller example 
(by accident) that also chrashes. Maybe I'll find the problem and can 
fix it once and for all :)


Regards,
Sven

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Martin Schreiber
On Sunday 03 March 2013 10:42:09 Sven Barth wrote:

 Do you mean compiling the compiler? If so than for real tests you SHOULD
 NOT compile the compiler by hand, but only through make cycle in the
 compiler directory or make all in the top level directory. This will
 ensure that the compiler is compiled correctly.

 And yes, I know the problem with -B and it's likely related to unit
 loading. This normally happens only in more complex projects like the
 compiler. Though I've just managed yesterday to create a smaller example
 (by accident) that also chrashes. Maybe I'll find the problem and can
 fix it once and for all :)

No, I mean MSEide. MSEide is a more complex application than the compiler it 
seems. :-)

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Sven Barth

On 03.03.2013 11:08, Martin Schreiber wrote:

On Sunday 03 March 2013 10:42:09 Sven Barth wrote:


Do you mean compiling the compiler? If so than for real tests you SHOULD
NOT compile the compiler by hand, but only through make cycle in the
compiler directory or make all in the top level directory. This will
ensure that the compiler is compiled correctly.

And yes, I know the problem with -B and it's likely related to unit
loading. This normally happens only in more complex projects like the
compiler. Though I've just managed yesterday to create a smaller example
(by accident) that also chrashes. Maybe I'll find the problem and can
fix it once and for all :)


No, I mean MSEide. MSEide is a more complex application than the compiler it
seems. :-)


No, it's not (at least not in that sense ;) ), because the compiler 
triggers this error as well, when I compile it inside the IDE.


Regards,
Sven

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Michael Van Canneyt



On Sun, 3 Mar 2013, Martin Schreiber wrote:


On Sunday 03 March 2013 08:12:28 Martin Schreiber wrote:

On Saturday 02 March 2013 11:28:25 Michael Van Canneyt wrote:

On Sat, 2 Mar 2013, Martin Schreiber wrote:

On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:

Anyway, what seems obvious from your numbers is that it is the linking
stage that needs speedup. This does not really come as a surprise.


On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll
check it later, currently I try to make MSEide+MSEgui Kylix3 compatible
so we have a benchmark on Linux too.


When you do, please send me the output of a strace -f


I will take a while because it is much work to make the code both FPC and
Delphi/Kylix compatible.


BTW, a significant percentage of the time is waiting for FPC compiling because
FPC normally crashes without -B.


A sure sign that you have fishy code :)
I have lots of code, and never ever experienced such a thing.
I even can't remember when the compiler last gave me a problem; and I use it 
daily.

Most problems I experience today have to do with version management.

I'm sorry, but I do not share your focus on speed. I have many projects, 
some of them have more units than mseide does, and yet I never have to wait 
more than a couple of seconds. The only exception is the Lazarus IDE.


If you say that mseide only compiles with -B, I would suggest looking for 
strange dependencies inside mseide, if you want to reduce compile time...


I do not want to say that FPC does not need speed improvements, but I do suspect you 
will get faster results looking at removing the -B requirement from mseide...


Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Martin Schreiber

Am 02.03.2013 11:11, schrieb Martin Schreiber:

On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:


Anyway, what seems obvious from your numbers is that it is the linking
stage that needs speedup. This does not really come as a surprise.


On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll check it
later, currently I try to make MSEide+MSEgui Kylix3 compatible so we have a
benchmark on Linux too.


I checked again, time of FPC linking stage is about 2.5 seconds.

Martin

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-03 Thread Marco van de Voort
In our previous episode, Michael Van Canneyt said:
 I'm sorry, but I do not share your focus on speed. I have many projects, 
 some of them have more units than mseide does, and yet I never have to wait 
 more than a couple of seconds. The only exception is the Lazarus IDE.

No, I have also seen it with Indy and the JCL. In the JCL case I tried to
isolate it even, (and managed to 3 or 4 units). 

And over the years, that are not the only cases.

 If you say that mseide only compiles with -B, I would suggest looking for 
 strange dependencies inside mseide, if you want to reduce compile time...

No, afaik there is certainly a problem in the compiler. Afaik I heard devs
that unit dependencies cycles with more than two units are not always
detected clearly (and probably that implies some hidden
interface-implementation dependencies?)
   
 I do not want to say that FPC does not need speed improvements, but I do 
 suspect you 
 will get faster results looking at removing the -B requirement from mseide...

That's for sure. 
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klämpfl
Am 01.03.2013 22:40, schrieb Martin Schreiber:
 On Friday 01 March 2013 22:30:31 Florian Klämpfl wrote:
 Am 01.03.2013 18:33, schrieb Martin Schreiber:
 Hi,

 In order to have a good benchmark for FPC

 Expect the next release to be even slower.

 Will it produce better code instead?


What means better? Faster? Less buggy? Smaller?

I did a quick test on win32, it seems to be a little bit smaller than 2.6.2:

02.03.2013  10:09 5.774.848 mseide.exe
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Martin Schreiber
On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:

 Anyway, what seems obvious from your numbers is that it is the linking
 stage that needs speedup. This does not really come as a surprise.

On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll check it 
later, currently I try to make MSEide+MSEgui Kylix3 compatible so we have a 
benchmark on Linux too.

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Martin Schreiber
On Saturday 02 March 2013 10:12:51 Florian Klämpfl wrote:
 Am 01.03.2013 22:40, schrieb Martin Schreiber:
  On Friday 01 March 2013 22:30:31 Florian Klämpfl wrote:
  Am 01.03.2013 18:33, schrieb Martin Schreiber:
  Hi,
 
  In order to have a good benchmark for FPC
 
  Expect the next release to be even slower.
 
  Will it produce better code instead?

 What means better? Faster? Less buggy? Smaller?

It means faster and smaller, Delphi 7 produced code is smaller and faster than 
FPC 2.6.2 code. A compiler must be bug free anyway. ;-)

 I did a quick test on win32, it seems to be a little bit smaller than
 2.6.2:

 02.03.2013  10:09 5.774.848 mseide.exe

Compiled with which FPC version?

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klämpfl
Am 02.03.2013 11:16, schrieb Martin Schreiber:
 On Saturday 02 March 2013 10:12:51 Florian Klämpfl wrote:
 Am 01.03.2013 22:40, schrieb Martin Schreiber:
 On Friday 01 March 2013 22:30:31 Florian Klämpfl wrote:
 Am 01.03.2013 18:33, schrieb Martin Schreiber:
 Hi,

 In order to have a good benchmark for FPC

 Expect the next release to be even slower.

 Will it produce better code instead?

 What means better? Faster? Less buggy? Smaller?

 It means faster and smaller, Delphi 7 produced code is smaller and faster 
 than 
 FPC 2.6.2 code. 

For i386-win32 maybe.

A compiler must be bug free anyway. ;-)
 
 I did a quick test on win32, it seems to be a little bit smaller than
 2.6.2:

 02.03.2013  10:09 5.774.848 mseide.exe
 
 Compiled with which FPC version?

trunk.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Martin Schreiber
On Saturday 02 March 2013 11:12:52 Florian Klämpfl wrote:

  I did a quick test on win32, it seems to be a little bit smaller than
  2.6.2:
 
  02.03.2013  10:09 5.774.848 mseide.exe
 
  Compiled with which FPC version?

 trunk.

??? MSEgui compiles with trunk?

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Sven Barth
Am 02.03.2013 11:15 schrieb Martin Schreiber mse00...@gmail.com:

 On Saturday 02 March 2013 11:12:52 Florian Klämpfl wrote:
 
   I did a quick test on win32, it seems to be a little bit smaller than
   2.6.2:
  
   02.03.2013  10:09 5.774.848 mseide.exe
  
   Compiled with which FPC version?
 
  trunk.

 ??? MSEgui compiles with trunk?

Surprise! :P

We don't try to respect backwards compatibility for nothing. ;)

Regards,
Sven
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Michael Van Canneyt



On Sat, 2 Mar 2013, Martin Schreiber wrote:


On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:


Anyway, what seems obvious from your numbers is that it is the linking
stage that needs speedup. This does not really come as a surprise.


On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll check it
later, currently I try to make MSEide+MSEgui Kylix3 compatible so we have a
benchmark on Linux too.


When you do, please send me the output of a strace -f

It will allow me to see what IO the Delphi/Kylix compiler does.

We can say for sure that the fact you use .pas as filename extension 
will cause FPC to do twice the number of stat() calls, because .pp is 
searched first...Logical therefore that the IO is slower.


(for example renaming all files to .pp takes off +/- 1 second here)

Nevertheless, I'd be interested in seeing the strace.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klämpfl
Am 02.03.2013 11:21, schrieb Martin Schreiber:
 On Saturday 02 March 2013 11:12:52 Florian Klämpfl wrote:

 I did a quick test on win32, it seems to be a little bit smaller than
 2.6.2:

 02.03.2013  10:09 5.774.848 mseide.exe

 Compiled with which FPC version?

 trunk.
 
 ??? MSEgui compiles with trunk?

Compiled with WPO and -O4 it starts and I can edit text and forms.
Nothing more tested.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klämpfl
Am 02.03.2013 11:28, schrieb Michael Van Canneyt:
 
 
 On Sat, 2 Mar 2013, Martin Schreiber wrote:
 
 On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:

 Anyway, what seems obvious from your numbers is that it is the linking
 stage that needs speedup. This does not really come as a surprise.

 On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll
 check it
 later, currently I try to make MSEide+MSEgui Kylix3 compatible so we
 have a
 benchmark on Linux too.
 
 When you do, please send me the output of a strace -f
 
 It will allow me to see what IO the Delphi/Kylix compiler does.
 
 We can say for sure that the fact you use .pas as filename extension
 will cause FPC to do twice the number of stat() calls, because .pp is
 searched first...Logical therefore that the IO is slower.
 
 (for example renaming all files to .pp takes off +/- 1 second here)
 
 Nevertheless, I'd be interested in seeing the strace.

Better parallize the building using some build units. Then it will be
probably compiled in less than 10 sec on a modern CPU.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Marco van de Voort
In our previous episode, Florian Kl?mpfl said:
  (for example renaming all files to .pp takes off +/- 1 second here)
  
  Nevertheless, I'd be interested in seeing the strace.
 
 Better parallize the building using some build units. Then it will be
 probably compiled in less than 10 sec on a modern CPU.

Better paralellize the compiler :-)
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Vittorio Giovara
 Am 01.03.2013 18:33, schrieb Martin Schreiber:
 Hi,

 In order to have a good benchmark for FPC

Afaik Delphi 7 is closed source, so inherently worse than any version of FPC.

Cheers,
Vittorio
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Marco van de Voort
In our previous episode, Martin Schreiber said:
  
   Compiled with which FPC version?
 
  trunk.
 
 ??? MSEgui compiles with trunk?

It is mostly unicodestring centric, so that shouldn't be such a surprise?
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Martin Schreiber
On Saturday 02 March 2013 14:01:10 Marco van de Voort wrote:
 In our previous episode, Martin Schreiber said:
Compiled with which FPC version?
  
   trunk.
 
  ??? MSEgui compiles with trunk?

 It is mostly unicodestring centric, so that shouldn't be such a surprise?

MSEgui has an own unicodestring manager on Linux which must be adapted to 
cpstrnew. Also the meaning of #nnn characters has changed and MSEgui uses 
some hacks with UnicodeStrings which must be adapted. I don't want to do the 
work before Unicode handling in FPC is finished and documented. And frankly, 
I don't believe in it anymore...

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Michael Van Canneyt



On Sat, 2 Mar 2013, Martin Schreiber wrote:


On Saturday 02 March 2013 14:01:10 Marco van de Voort wrote:

In our previous episode, Martin Schreiber said:

Compiled with which FPC version?


trunk.


??? MSEgui compiles with trunk?


It is mostly unicodestring centric, so that shouldn't be such a surprise?


MSEgui has an own unicodestring manager on Linux which must be adapted to
cpstrnew. Also the meaning of #nnn characters has changed and MSEgui uses
some hacks with UnicodeStrings which must be adapted. I don't want to do the
work before Unicode handling in FPC is finished and documented. And frankly,
I don't believe in it anymore...


Clearly, you have not been watching SVN, because Paul Ishenin is currently 
integrating the native Object Pascal unicode string manager in trunk.


Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Craig Peterson


On Mar 2, 2013, at 3:52 AM, Michael Van Canneyt mich...@freepascal.org wrote:

 If you hire 2 painters to paint the whole of your house,
 and one doesn't paint the inside, because you don't see it from the 
 outside, of course he will be finished faster; he didn't perform the same 
 work.

Delphi is generating enough information that you can debug using it.  The fact 
that its debugger is built in and FPC requires an external debugger that can't 
read the .ppus is an implementation detail.  A more apt comparison would be 
that we hired the painters to paint the outside of the house and one of them 
did the inside too because his tools require him to paint both sides of the 
walls.

-- 
Craig Peterson
Scooter Software
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klaempfl

Am 02.03.2013 13:24, schrieb Marco van de Voort:

In our previous episode, Florian Kl?mpfl said:

(for example renaming all files to .pp takes off +/- 1 second here)

Nevertheless, I'd be interested in seeing the strace.


Better parallize the building using some build units. Then it will be
probably compiled in less than 10 sec on a modern CPU.


Better paralellize the compiler :-)


In theory yes but I still fear that the threadvars and synchronization 
eats much of the advantage in this case.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klaempfl

Am 02.03.2013 15:15, schrieb Craig Peterson:



On Mar 2, 2013, at 3:52 AM, Michael Van Canneyt
mich...@freepascal.org wrote:


If you hire 2 painters to paint the whole of your house, and one
doesn't paint the inside, because you don't see it from the
outside, of course he will be finished faster; he didn't perform
the same work.


Delphi is generating enough information that you can debug using it.
The fact that its debugger is built in and FPC requires an external
debugger that can't read the .ppus is an implementation detail.  A
more apt comparison would be that we hired the painters to paint the
outside of the house and one of them did the inside too because his
tools require him to paint both sides of the walls.



While the other one can paint only in one color and only buildings with 
one floor ;)

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Marco van de Voort
In our previous episode, Florian Klaempfl said:
  Better parallize the building using some build units. Then it will be
  probably compiled in less than 10 sec on a modern CPU.
 
  Better paralellize the compiler :-)
 
 In theory yes but I still fear that the threadvars and synchronization 
 eats much of the advantage in this case.

I don't see why there would be more synchronization overhead than make?
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klaempfl

Am 02.03.2013 16:23, schrieb Marco van de Voort:

In our previous episode, Florian Klaempfl said:

Better parallize the building using some build units. Then it will be
probably compiled in less than 10 sec on a modern CPU.


Better paralellize the compiler :-)


In theory yes but I still fear that the threadvars and synchronization
eats much of the advantage in this case.


I don't see why there would be more synchronization overhead than make?


In a parallelized compiler symtables etc. might be shared and this might 
require a lot of synchronization to prevent data corruption.
With make, the different units can be seperated much better by a human 
than the compiler can do.


Not to mention the huge effort it would be to get the compiler 
parallelized internally.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Henry Vermaak
On Sat, Mar 02, 2013 at 04:23:32PM +0100, Marco van de Voort wrote:
 In our previous episode, Florian Klaempfl said:
   Better parallize the building using some build units. Then it will be
   probably compiled in less than 10 sec on a modern CPU.
  
   Better paralellize the compiler :-)
  
  In theory yes but I still fear that the threadvars and synchronization 
  eats much of the advantage in this case.
 
 I don't see why there would be more synchronization overhead than make?

So why not keep it simple and let the build system handle parallel jobs?

Henry
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Michael Van Canneyt



On Sat, 2 Mar 2013, Craig Peterson wrote:




On Mar 2, 2013, at 3:52 AM, Michael Van Canneyt mich...@freepascal.org wrote:


If you hire 2 painters to paint the whole of your house,
and one doesn't paint the inside, because you don't see it from the outside, 
of course he will be finished faster; he didn't perform the same work.


Delphi is generating enough information that you can debug using it. 
The fact that its debugger is built in and FPC requires an external debugger 
that can't read the .ppus is an implementation detail.


You cannot ignore the implementation details. Making abstraction of these 
details
means ignoring the benefits that follow from these details:

It is exactly these details that allow FPC to support so many platforms,
and why Delphi is stuck with Windows and remote compiling/debugging for MaCOS.
Delphi does not even run on these platforms ! FPC does.

If you are - like Martin, I suspect - not interested in weird and exotic 
platforms
then of course FPC's implementation choice and the attached consequences present you 
with a disadvantage because of these 'implementation details'.


But if you are interested in ARM, raspberry PI, MIPS and whatever there is out 
there,
it is these implementation details that make FPC the only possible choice.

All this said: 
You will not hear me claiming that there is no room for improvement in FPC.
I'm sure there is. However, I do not think we'll be able to match Delphi's 
speed without sacrificing the main goal of FPC: to support as much platforms 
as possible.


So, proposals for improvement are most welcome.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Marco van de Voort
In our previous episode, Florian Klaempfl said:
  I don't see why there would be more synchronization overhead than make?
 
 In a parallelized compiler symtables etc. might be shared and this might 
 require a lot of synchronization to prevent data corruption.
 With make, the different units can be seperated much better by a human 
 than the compiler can do.

Only if you want to avoid multiply used units to be only once in memory. 
IMHO if you are afraid of that, then don't do that :-)
 
 Not to mention the huge effort it would be to get the compiler 
 parallelized internally.

As said the first step can be relatively simple. Get rid of globals some
more, and then only the parts before compile() needs to be changed. The rest
more or less runs as is, and the linker remains single threaded after all
compile threads have finished?
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Marco van de Voort
In our previous episode, Henry Vermaak said:
  I don't see why there would be more synchronization overhead than make?
 
 So why not keep it simple and let the build system handle parallel jobs?

I like autobuilding. IMHO that is a core attraction of unit based (modular)
languages.

Manually maintaining dependencies between compilation units is stone-age.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klaempfl

Am 02.03.2013 17:24, schrieb Marco van de Voort:

In our previous episode, Florian Klaempfl said:

I don't see why there would be more synchronization overhead than make?


In a parallelized compiler symtables etc. might be shared and this might
require a lot of synchronization to prevent data corruption.
With make, the different units can be seperated much better by a human
than the compiler can do.


Only if you want to avoid multiply used units to be only once in memory.
IMHO if you are afraid of that, then don't do that :-)


Forking compilers on a by unit level instead of some meta level is 
typically slower because reading ppus takes a lot of time.





Not to mention the huge effort it would be to get the compiler
parallelized internally.


As said the first step can be relatively simple. Get rid of globals some
more, and then only the parts before compile() needs to be changed.


In theory, yes. In practice this is a huge effort. Just one example: 
inlining requires that the node tree of a procedure is copied. However, 
during copying the tree is annotated for proper copying. Bottom line: 
even if the compiler finished to compile a unit, the generated info is 
not read only.


___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Florian Klaempfl

Am 02.03.2013 16:49, schrieb Michael Van Canneyt:

All this said: You will not hear me claiming that there is no room for
improvement in FPC.
I'm sure there is. However, I do not think we'll be able to match
Delphi's speed without sacrificing the main goal of FPC: to support as
much platforms as possible.


Well, not to mention the goal of maintainability.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Graeme Geldenhuys
On 2013-03-02 10:28, Michael Van Canneyt wrote:
 We can say for sure that the fact you use .pas as filename extension 
 will cause FPC to do twice the number of stat() calls, because .pp is 
 searched first...Logical therefore that the IO is slower.


Second time I hear this this week. Can we (in our own copies of FPC)
change this to search .pas first? If so, where in the source?


Regards,
  - Graeme -

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Michael Van Canneyt



On Sat, 2 Mar 2013, Graeme Geldenhuys wrote:


On 2013-03-02 10:28, Michael Van Canneyt wrote:

We can say for sure that the fact you use .pas as filename extension
will cause FPC to do twice the number of stat() calls, because .pp is
searched first...Logical therefore that the IO is slower.



Second time I hear this this week. Can we (in our own copies of FPC)
change this to search .pas first? If so, where in the source?


Search for sourceext and pasext.

fppu.pas:   Found:=UnitExists(sourceext,hs);
fppu.pas:  
Message1(unit_t_unitsearch,ChangeFileExt(sourcefn,sourceext));
fppu.pas:
fnd:=FindFile(ChangeFileExt(sourcefn,sourceext),'',true,hs);
globals.pas:   sourceext  = '.pp';
options.pas:  if 
FileExists(inputfilepath+ChangeFileExt(inputfilename,sourceext)) then
options.pas:inputfilename:=ChangeFileExt(inputfilename,sourceext)
scanner.pas:   
found:=findincludefile(path,ChangeFileExt(name,sourceext),foundfile);

Be careful when changing this, because the compiler will then also search for rtl/fcl/package 
source files with different names, which may result in nasty surprises.


Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Martin Schreiber
On Saturday 02 March 2013 11:28:25 Michael Van Canneyt wrote:
 On Sat, 2 Mar 2013, Martin Schreiber wrote:
  On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:
  Anyway, what seems obvious from your numbers is that it is the linking
  stage that needs speedup. This does not really come as a surprise.
 
  On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll
  check it later, currently I try to make MSEide+MSEgui Kylix3 compatible
  so we have a benchmark on Linux too.

 When you do, please send me the output of a strace -f

I will take a while because it is much work to make the code both FPC and 
Delphi/Kylix compatible.

 It will allow me to see what IO the Delphi/Kylix compiler does.

 We can say for sure that the fact you use .pas as filename extension
 will cause FPC to do twice the number of stat() calls, because .pp is
 searched first...Logical therefore that the IO is slower.

AFAIK *.pas is Delphi compatible? ;-)

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-02 Thread Martin Schreiber
On Sunday 03 March 2013 08:12:28 Martin Schreiber wrote:
 On Saturday 02 March 2013 11:28:25 Michael Van Canneyt wrote:
  On Sat, 2 Mar 2013, Martin Schreiber wrote:
   On Saturday 02 March 2013 10:52:18 Michael Van Canneyt wrote:
   Anyway, what seems obvious from your numbers is that it is the linking
   stage that needs speedup. This does not really come as a surprise.
  
   On Windows FPC linking time of MSEide is about 2-3 seconds IIRC, I'll
   check it later, currently I try to make MSEide+MSEgui Kylix3 compatible
   so we have a benchmark on Linux too.
 
  When you do, please send me the output of a strace -f

 I will take a while because it is much work to make the code both FPC and
 Delphi/Kylix compatible.

BTW, a significant percentage of the time is waiting for FPC compiling because 
FPC normally crashes without -B.

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


[fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Martin Schreiber

Hi,

In order to have a good benchmark for FPC I made MSEide+MSEgui Delphi 7 
compatible again.

Source (master branch 2400be4999b254dbf335e4777baa33b13ea72649):
http://gitorious.org/mseide-msegui

As testcase I used compiling MSEide, all dcu's, ppu's and o's deleted 
before compiling.

System:
win2000, AMD Atlon XP 3000+, 1GB RAM

Command line Delphi 7:
dcc32 -B -D+ -I..\..\lib\common\kernel -U..\..\lib\common\kernel 
-U..\..\lib\common\audio -U..\..\lib\common\opengl 
-U..\..\lib\common\report -U..\..\lib\common\db 
-U..\..\lib\common\crypto -U..\..\lib\common\graphics 
-U..\..\lib\common\i18n -U..\..\lib\common\kernel\windows 
-U..\..\lib\common\image -U..\..\lib\common\widgets 
-U..\..\lib\common\designutils -U..\..\lib\common\sysutils 
-U..\..\lib\common\editwidgets -U..\..\lib\common\dialogs 
-U..\..\lib\common\regcomponents -U..\..\lib\common\serialcomm 
-U..\..\lib\common\printer -U..\..\lib\common\ifi 
-U..\..\lib\common\math -U..\..\lib\common\delphicompatibility 
-U..\..\lib\common\fpccompatibility mseide.pas


Result Delphi 7:
483086 lines, 6.72 seconds, 3396292 bytes code, 1217961 bytes data.

F:\proj\mseide-msegui\apps\idedir mseide.exe
 Datenträger in Laufwerk F: ist win2000_F
 Datenträgernummer: 5C58-C4EF

 Verzeichnis von F:\proj\mseide-msegui\apps\ide

01.03.2013  17:225'062'144 mseide.exe

Command line FPC 2.6.2:
ppc386.exe -g -Xg -O2 -CX -XX -Xs -B -Fi..\..\lib\common\kernel 
-Fu..\..\lib\common\kernel -Fu..\..\lib\common\audio 
-Fu..\..\lib\common\opengl -Fu..\..\lib\common\report 
-Fu..\..\lib\common\db -Fu..\..\lib\common\crypto 
-Fu..\..\lib\common\graphics -Fu..\..\lib\common\i18n 
-Fu..\..\lib\common\kernel\windows -Fu..\..\lib\common\image 
-Fu..\..\lib\common\widgets -Fu..\..\lib\common\designutils 
-Fu..\..\lib\common\sysutils -Fu..\..\lib\common\editwidgets 
-Fu..\..\lib\common\dialogs -Fu..\..\lib\common\regcomponents 
-Fu..\..\lib\common\serialcomm -Fu..\..\lib\common\printer 
-Fu..\..\lib\common\ifi -Fu..\..\lib\common\math 
-Fu..\..\lib\common\delphicompatibility 
-Fu..\..\lib\common\fpccompatibility -omseidefp.exe mseide.pas


Result FPC 2.6.2:
489756 lines compiled, 84.6 sec , 3309520 bytes code, 2691084 bytes data

F:\proj\mseide-msegui\apps\idedir mseidefp.exe
 Datenträger in Laufwerk F: ist win2000_F
 Datenträgernummer: 5C58-C4EF

 Verzeichnis von F:\proj\mseide-msegui\apps\ide

01.03.2013  17:246'026'259 mseidefp.exe

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Michael Van Canneyt



On Fri, 1 Mar 2013, Martin Schreiber wrote:


Hi,

In order to have a good benchmark for FPC I made MSEide+MSEgui Delphi 7 
compatible again.

Source (master branch 2400be4999b254dbf335e4777baa33b13ea72649):
http://gitorious.org/mseide-msegui


For a correct test, you should not enable debug information in FPC.
Or enable generation of turbo debug information and the corresponding linker 
map in Delphi.

Otherwise you are comparing apples with pears, as -D+ in delphi does not do nearly the same 
thing as -g in FPC.


That said, it takes about 30 seconds on Linux on a laptop to do the same thing.
Intel(R) Core(TM)2 Duo CPU T8300  @ 2.40GHz
(A centrino - whatever all these numbers mean today)

Of this, about 8 seconds are spent in the gnu linker ld.
Taking away smartlinking removes 10 seconds from the total time.

So, at least we can say with confidence that the linker takes a
large part of the time.

I am not sure if the Windows version of FPC uses ld when smartlinking, 
so comparison should be done with caution.


On linux, there are a lot of getdents and stat calls:
823 getdents calls.
14611 stat calls.
So I'm sure there is room for some improvement there.

What I remember from windows 2000 is that FPC/ld/etc is several factors 
slower on the same hardware as when run under Linux.


Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Michael Van Canneyt



On Fri, 1 Mar 2013, Michael Van Canneyt wrote:




On Fri, 1 Mar 2013, Martin Schreiber wrote:


Hi,

In order to have a good benchmark for FPC I made MSEide+MSEgui Delphi 7 
compatible again.

Source (master branch 2400be4999b254dbf335e4777baa33b13ea72649):
http://gitorious.org/mseide-msegui


For a correct test, you should not enable debug information in FPC.
Or enable generation of turbo debug information and the corresponding linker 
map in Delphi.


Otherwise you are comparing apples with pears, as -D+ in delphi does not do 
nearly the same thing as -g in FPC.


That said, it takes about 30 seconds on Linux on a laptop to do the same 
thing.


Forgot to say that this is done using the 64-bit, unoptimized version of FPC, 
which is probably slow when compared to the 32-bit optimized version of FPC.


Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Marco van de Voort
In our previous episode, Michael Van Canneyt said:
 What I remember from windows 2000 is that FPC/ld/etc is several factors 
 slower on the same hardware as when run under Linux.

Afaik the internal linker is default on windows since 2.2.0 ?

There are still some issues with external .a archives, which is why the
makefiles afaik enable the external linker for the textmode IDE (libgdb)
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Michael Van Canneyt



On Fri, 1 Mar 2013, Marco van de Voort wrote:


In our previous episode, Michael Van Canneyt said:

What I remember from windows 2000 is that FPC/ld/etc is several factors
slower on the same hardware as when run under Linux.


Afaik the internal linker is default on windows since 2.2.0 ?


Also when smartlinking ? I am not sure about this.

Michael.
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Marco van de Voort
In our previous episode, Michael Van Canneyt said:
  In our previous episode, Michael Van Canneyt said:
  What I remember from windows 2000 is that FPC/ld/etc is several factors
  slower on the same hardware as when run under Linux.
 
  Afaik the internal linker is default on windows since 2.2.0 ?
 
 Also when smartlinking ? 

I think that was one of the primary reasons for it even.

Before the internal linker, smartlinking lazarus (every unit in it) with LD
could eat 1.5GB

On *nix, sections smartlinking remedied that somewhat, but on Windows not.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Florian Klämpfl
Am 01.03.2013 18:33, schrieb Martin Schreiber:
 Hi,
 
 In order to have a good benchmark for FPC

Expect the next release to be even slower.

___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel


Re: [fpc-devel] Comparison FPC 2.6.2 - Delphi 7

2013-03-01 Thread Martin Schreiber
On Friday 01 March 2013 22:30:31 Florian Klämpfl wrote:
 Am 01.03.2013 18:33, schrieb Martin Schreiber:
  Hi,
 
  In order to have a good benchmark for FPC

 Expect the next release to be even slower.

Will it produce better code instead?

Martin
___
fpc-devel maillist  -  fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel