Re: Pass multiple same options to rtems_linkflags

[Peng Fan]

Hi,


2014-07-26 9:28 GMT+08:00 Chris Johns :

> On 26/07/2014 12:25 am, Peng Fan wrote:
>
>> Hi Chris,
>>
>> I build a rap file using such a command :
>> rtems-ld --lib-path
>> /opt/rtems-4.11/arm-rtems4.11/realview_pbx_a9_qemu/lib --lib m --lib
>> rtemscpu --lib rtemsbsp --base rtld.prelink --entry  rtems a.o b.o c.o
>> *.o ...
>> Is this ok? can reference a symbol in librtemscpu.a librtemsbsp.a? or
>> the reference symbol from librtemscpu.a librtemsbsp.a should be included
>> in the base image but not in the rap file?
>>
>
> This is fine. You do not need to load a base image with everything you
> might need.
>
> If you create another RAP file and do the same thing and that RAP pulls
> the same code in from one of those libraries it will not be linked to.
> Rather the first instance of the code loaded is used. The downside is a
> possible waste of code.
>
Yeah.Other RAP file which references the same code that already in the
first rap should not pull the same code into the final image.

>
> I suppose we could add code to compact the memory and not loaded the
> object file and so the overhead is limited to the RAP file.

Sorry. I can not got this. what code should be added to rtl-host?

>
>
>
>> I set rtems_linkflags like this:
>>  rtems_linkflags = ['--base', 'rtld.prelink',
>> '--lib-path',
>> '/opt/rtems-4.11/arm-rtems4.11/realview_pbx_a9_qemu/lib',
>> '--entry', 'my_main', '--lib', 'm', '--lib',
>> 'rtemscpu' , '--lib', 'rtemsbsp'],
>> But waf only uses one '--lib', it removes the other two '--lib', like
>> this: xxx '--entry', 'my_main', '--lib', 'm', 'rtemscpu', 'rtemsbsp',xxx
>> I am not familar with waf, any advice about his?
>>
>
> Try using LIBS and adding to that. Getting the order correct can be tricky.

To bld whose feature is 'c program', libs = ['m', 'xxx'] works. But to 'c
rap' which is defined in RTL, adding libs to bld of waf does not work. So I
wrote a patch to fix this, the patch first clear the RTEMS_LINKFLAGS env,
then refill it. The patch is in the attachments.

Regards,
Peng.


>
> Chris
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Pass multiple same options to rtems_linkflags

[Peng Fan]

Sorry. The patch is in the attachment. Missed it just now.


2014-07-27 22:00 GMT+08:00 Peng Fan :

> Hi,
>
>
> 2014-07-26 9:28 GMT+08:00 Chris Johns :
>
> On 26/07/2014 12:25 am, Peng Fan wrote:
>>
>>> Hi Chris,
>>>
>>> I build a rap file using such a command :
>>> rtems-ld --lib-path
>>> /opt/rtems-4.11/arm-rtems4.11/realview_pbx_a9_qemu/lib --lib m --lib
>>> rtemscpu --lib rtemsbsp --base rtld.prelink --entry  rtems a.o b.o c.o
>>> *.o ...
>>> Is this ok? can reference a symbol in librtemscpu.a librtemsbsp.a? or
>>> the reference symbol from librtemscpu.a librtemsbsp.a should be included
>>> in the base image but not in the rap file?
>>>
>>
>> This is fine. You do not need to load a base image with everything you
>> might need.
>>
>> If you create another RAP file and do the same thing and that RAP pulls
>> the same code in from one of those libraries it will not be linked to.
>> Rather the first instance of the code loaded is used. The downside is a
>> possible waste of code.
>>
> Yeah.Other RAP file which references the same code that already in the
> first rap should not pull the same code into the final image.
>
>>
>> I suppose we could add code to compact the memory and not loaded the
>> object file and so the overhead is limited to the RAP file.
>
> Sorry. I can not got this. what code should be added to rtl-host?
>
>>
>>
>>
>>> I set rtems_linkflags like this:
>>>  rtems_linkflags = ['--base', 'rtld.prelink',
>>> '--lib-path',
>>> '/opt/rtems-4.11/arm-rtems4.11/realview_pbx_a9_qemu/lib',
>>> '--entry', 'my_main', '--lib', 'm', '--lib',
>>> 'rtemscpu' , '--lib', 'rtemsbsp'],
>>> But waf only uses one '--lib', it removes the other two '--lib', like
>>> this: xxx '--entry', 'my_main', '--lib', 'm', 'rtemscpu', 'rtemsbsp',xxx
>>> I am not familar with waf, any advice about his?
>>>
>>
>> Try using LIBS and adding to that. Getting the order correct can be
>> tricky.
>
> To bld whose feature is 'c program', libs = ['m', 'xxx'] works. But to 'c
> rap' which is defined in RTL, adding libs to bld of waf does not work. So I
> wrote a patch to fix this, the patch first clear the RTEMS_LINKFLAGS env,
> then refill it. The patch is in the attachments.
>
> Regards,
> Peng.
>
>
>>
>> Chris
>>
>
>


0001-Fix-multiple-same-options-in-rtems_linkflags-problem.patch
Description: Binary data
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Pass multiple same options to rtems_linkflags

[Chris Johns]

On 28/07/2014 12:01 am, Peng Fan wrote:

Sorry. The patch is in the attachment. Missed it just now.


The patch looks fine and it is nice solution. Maybe the RTL parts should 
be merged into the git://git.rtems.org/chrisj/rtems_waf.git repo and the 
rtl.git repo made to reference this repo. The examples-v2 repo does this 
now. Are you able to look into this for me ?





2014-07-27 22:00 GMT+08:00 Peng Fan mailto:van.free...@gmail.com>>:

Hi,


2014-07-26 9:28 GMT+08:00 Chris Johns mailto:chr...@rtems.org>>:

On 26/07/2014 12:25 am, Peng Fan wrote:

Hi Chris,

I build a rap file using such a command :
rtems-ld --lib-path
/opt/rtems-4.11/arm-rtems4.11/__realview_pbx_a9_qemu/lib
--lib m --lib
rtemscpu --lib rtemsbsp --base rtld.prelink --entry  rtems
a.o b.o c.o
*.o ...
Is this ok? can reference a symbol in librtemscpu.a
librtemsbsp.a? or
the reference symbol from librtemscpu.a librtemsbsp.a should
be included
in the base image but not in the rap file?


This is fine. You do not need to load a base image with
everything you might need.

If you create another RAP file and do the same thing and that
RAP pulls the same code in from one of those libraries it will
not be linked to. Rather the first instance of the code loaded
is used. The downside is a possible waste of code.

Yeah.Other RAP file which references the same code that already in
the first rap should not pull the same code into the final image.


I suppose we could add code to compact the memory and not loaded
the object file and so the overhead is limited to the RAP file.

Sorry. I can not got this. what code should be added to rtl-host?



Thinking about this some more I can understand why you did not get it 
and also your question about host side is a good one. Thinking out loud ...


Lets say we have RAP module A and B and both reference 
'rtems_rate_monotonic_get_status' which is not resident in the base 
kernel image. Both RAP modules will get a copy of the object file. We 
load A first and it's copy is fixed up and 
'rtems_rate_monotonic_get_status' is placed in the global symbol table. 
Any other global symbols in that object file are also placed in the 
global symbol table. Then we load B and we see 
'rtems_rate_monotonic_get_status' is present in the global symbol table. 
I suspect a duplicate symbol error is raised because we currently do not 
know if both versions of the symbol match the same code. I suppose a 
CRC16 could be added to the object file's data and if A and B's versions 
match we ignore B's and the global symbols can be referenced.


If we can determine the same is code is present I suspect removing the 
unreferenced code in B at load time may be difficult on the target 
because we have merged the object file's sections together with all the 
other object files in the RAP file and may not have the required info 
present to strip it out on target.


On the host side is the '--runtime-lib' (-P) option of rtems-ld doing 
this anyway so why do we need to bother with the above ?


We are in need of user documentation for the RTL code.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Pass multiple same options to rtems_linkflags

[Peng Fan]

Hi,


2014-07-28 9:13 GMT+08:00 Chris Johns :

> On 28/07/2014 12:01 am, Peng Fan wrote:
>
>> Sorry. The patch is in the attachment. Missed it just now.
>>
>
> The patch looks fine and it is nice solution. Maybe the RTL parts should
> be merged into the git://git.rtems.org/chrisj/rtems_waf.git repo and the
> rtl.git repo made to reference this repo. The examples-v2 repo does this
> now. Are you able to look into this for me ?
>
You mean using waf to compile rtems applications? Yeah, I saw examples-v2
use waf wscript.

Why merge RTL to rtems_waf.git? Provide a more convinent way to let user
use RTL?
I am glad if I can do something. Before that I prefer your advice :).


>
>
>>
>> 2014-07-27 22:00 GMT+08:00 Peng Fan > >:
>>
>>
>> Hi,
>>
>>
>> 2014-07-26 9:28 GMT+08:00 Chris Johns > >:
>>
>>
>> On 26/07/2014 12:25 am, Peng Fan wrote:
>>
>> Hi Chris,
>>
>> I build a rap file using such a command :
>> rtems-ld --lib-path
>> /opt/rtems-4.11/arm-rtems4.11/__realview_pbx_a9_qemu/lib
>>
>> --lib m --lib
>> rtemscpu --lib rtemsbsp --base rtld.prelink --entry  rtems
>> a.o b.o c.o
>> *.o ...
>> Is this ok? can reference a symbol in librtemscpu.a
>> librtemsbsp.a? or
>> the reference symbol from librtemscpu.a librtemsbsp.a should
>> be included
>> in the base image but not in the rap file?
>>
>>
>> This is fine. You do not need to load a base image with
>> everything you might need.
>>
>> If you create another RAP file and do the same thing and that
>> RAP pulls the same code in from one of those libraries it will
>> not be linked to. Rather the first instance of the code loaded
>> is used. The downside is a possible waste of code.
>>
>> Yeah.Other RAP file which references the same code that already in
>> the first rap should not pull the same code into the final image.
>>
>>
>> I suppose we could add code to compact the memory and not loaded
>> the object file and so the overhead is limited to the RAP file.
>>
>> Sorry. I can not got this. what code should be added to rtl-host?
>>
>>
> Thinking about this some more I can understand why you did not get it and
> also your question about host side is a good one. Thinking out loud ...
>
> Lets say we have RAP module A and B and both reference
> 'rtems_rate_monotonic_get_status' which is not resident in the base
> kernel image. Both RAP modules will get a copy of the object file. We load
> A first and it's copy is fixed up and 'rtems_rate_monotonic_get_status'
> is placed in the global symbol table. Any other global symbols in that
> object file are also placed in the global symbol table. Then we load B and
> we see 'rtems_rate_monotonic_get_status' is present in the global symbol
> table. I suspect a duplicate symbol error is raised because we currently do
> not know if both versions of the symbol match the same code. I suppose a
> CRC16 could be added to the object file's data and if A and B's versions
> match we ignore B's and the global symbols can be referenced.
>
> If we can determine the same is code is present I suspect removing the
> unreferenced code in B at load time may be difficult on the target because
> we have merged the object file's sections together with all the other
> object files in the RAP file and may not have the required info present to
> strip it out on target.
>
> On the host side is the '--runtime-lib' (-P) option of rtems-ld doing this
> anyway so why do we need to bother with the above ?
>
> Yeah. --runtime-lib handles the common code used by multiple RAP files.

> We are in need of user documentation for the RTL code.

Hah! what kind of doc do you prefer? doxgen doc in patch format or just
wiki?  And the documentation is about how to let user can easily integrate
RTL into his/her application?

Currently, I am more concerned about another problem which we talked about
when I load python rap and you also talked about with sebh.
lets say that we have a.o b.o c.o and the three .o files references symbols
in libc.a libm.a librtemscpu.a librtemsbsp.a.
Because libc.a libm.a librtemscpu.a librtemsbsp.a is not compile with
-mlong-calls, so if the rap file is big enough, RTL target may fail to load
the rap file since reloc entry from libxx.a is near jump, but dest symbol
is in far away.
I am hacking it these few days, but still do not have a good idea, because
it is hard to convert reloc entry in libxxx.a from near reference to far
reference as '-mlong-call' does.

Regards,
Peng.

>
>
> Chris
>
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: Pass multiple same options to rtems_linkflags

[Chris Johns]

On 28/07/2014 11:22 pm, Peng Fan wrote:

Hi,


2014-07-28 9:13 GMT+08:00 Chris Johns mailto:chr...@rtems.org>>:

On 28/07/2014 12:01 am, Peng Fan wrote:

Sorry. The patch is in the attachment. Missed it just now.


The patch looks fine and it is nice solution. Maybe the RTL parts
should be merged into the git://git.rtems.org/chrisj/__rtems_waf.git
 repo and the rtl.git
repo made to reference this repo. The examples-v2 repo does this
now. Are you able to look into this for me ?

You mean using waf to compile rtems applications? Yeah, I saw
examples-v2 use waf wscript.

Why merge RTL to rtems_waf.git? Provide a more convinent way to let user
use RTL?
I am glad if I can do something. Before that I prefer your advice :).


I mean merge the RTL support in the rtems.py file to the rtems_waf.git repo.






2014-07-27 22:00 GMT+08:00 Peng Fan mailto:van.free...@gmail.com>
>__>:


 Hi,


 2014-07-26 9:28 GMT+08:00 Chris Johns mailto:chr...@rtems.org>
 >>:


 On 26/07/2014 12:25 am, Peng Fan wrote:

 Hi Chris,

 I build a rap file using such a command :
 rtems-ld --lib-path

/opt/rtems-4.11/arm-rtems4.11/realview_pbx_a9_qemu/lib

 --lib m --lib
 rtemscpu --lib rtemsbsp --base rtld.prelink --entry
  rtems
 a.o b.o c.o
 *.o ...
 Is this ok? can reference a symbol in librtemscpu.a
 librtemsbsp.a? or
 the reference symbol from librtemscpu.a
librtemsbsp.a should
 be included
 in the base image but not in the rap file?


 This is fine. You do not need to load a base image with
 everything you might need.

 If you create another RAP file and do the same thing
and that
 RAP pulls the same code in from one of those libraries
it will
 not be linked to. Rather the first instance of the code
loaded
 is used. The downside is a possible waste of code.

 Yeah.Other RAP file which references the same code that
already in
 the first rap should not pull the same code into the final
image.


 I suppose we could add code to compact the memory and
not loaded
 the object file and so the overhead is limited to the
RAP file.

 Sorry. I can not got this. what code should be added to
rtl-host?


Thinking about this some more I can understand why you did not get
it and also your question about host side is a good one. Thinking
out loud ...

Lets say we have RAP module A and B and both reference
'rtems_rate_monotonic_get___status' which is not resident in the
base kernel image. Both RAP modules will get a copy of the object
file. We load A first and it's copy is fixed up and
'rtems_rate_monotonic_get___status' is placed in the global symbol
table. Any other global symbols in that object file are also placed
in the global symbol table. Then we load B and we see
'rtems_rate_monotonic_get___status' is present in the global symbol
table. I suspect a duplicate symbol error is raised because we
currently do not know if both versions of the symbol match the same
code. I suppose a CRC16 could be added to the object file's data and
if A and B's versions match we ignore B's and the global symbols can
be referenced.

If we can determine the same is code is present I suspect removing
the unreferenced code in B at load time may be difficult on the
target because we have merged the object file's sections together
with all the other object files in the RAP file and may not have the
required info present to strip it out on target.

On the host side is the '--runtime-lib' (-P) option of rtems-ld
doing this anyway so why do we need to bother with the above ?

Yeah. --runtime-lib handles the common code used by multiple RAP files.



Great.


We are in need of user documentation for the RTL code.

Hah! what kind of doc do you prefer? doxgen doc in patch format or just
wiki?  And the documentation is about how to let user can easily
integrate RTL into his/her application?


Yes, something about how to use the RTL, rtems-ld and what happens with 
applications.




Currently, I am more concerned about another problem which we talked
about when I load python rap and you also talked about with sebh.
lets say that we have a.o b.o c.o and the three .o files references
symbols in libc.a libm.a librtemscpu.a librtemsbsp.a.
Because libc.a li

Re: Pass multiple same options to rtems_linkflags

[Peng Fan]

2014-07-28 21:32 GMT+08:00 Chris Johns :

> On 28/07/2014 11:22 pm, Peng Fan wrote:
>
>> Hi,
>>
>>
>> 2014-07-28 9:13 GMT+08:00 Chris Johns > >:
>>
>>
>> On 28/07/2014 12:01 am, Peng Fan wrote:
>>
>> Sorry. The patch is in the attachment. Missed it just now.
>>
>>
>> The patch looks fine and it is nice solution. Maybe the RTL parts
>> should be merged into the git://git.rtems.org/chrisj/__rtems_waf.git
>>  repo and the rtl.git
>>
>> repo made to reference this repo. The examples-v2 repo does this
>> now. Are you able to look into this for me ?
>>
>> You mean using waf to compile rtems applications? Yeah, I saw
>> examples-v2 use waf wscript.
>>
>> Why merge RTL to rtems_waf.git? Provide a more convinent way to let user
>> use RTL?
>> I am glad if I can do something. Before that I prefer your advice :).
>>
>
> I mean merge the RTL support in the rtems.py file to the rtems_waf.git
> repo.
>
>
>>
>>
>>
>> 2014-07-27 22:00 GMT+08:00 Peng Fan > 
>> >__>:
>>
>>
>>
>>  Hi,
>>
>>
>>  2014-07-26 9:28 GMT+08:00 Chris Johns > 
>>  >>:
>>
>>
>>
>>  On 26/07/2014 12:25 am, Peng Fan wrote:
>>
>>  Hi Chris,
>>
>>  I build a rap file using such a command :
>>  rtems-ld --lib-path
>>
>> /opt/rtems-4.11/arm-rtems4.11/realview_pbx_a9_qemu/lib
>>
>>
>>  --lib m --lib
>>  rtemscpu --lib rtemsbsp --base rtld.prelink --entry
>>   rtems
>>  a.o b.o c.o
>>  *.o ...
>>  Is this ok? can reference a symbol in librtemscpu.a
>>  librtemsbsp.a? or
>>  the reference symbol from librtemscpu.a
>> librtemsbsp.a should
>>  be included
>>  in the base image but not in the rap file?
>>
>>
>>  This is fine. You do not need to load a base image with
>>  everything you might need.
>>
>>  If you create another RAP file and do the same thing
>> and that
>>  RAP pulls the same code in from one of those libraries
>> it will
>>  not be linked to. Rather the first instance of the code
>> loaded
>>  is used. The downside is a possible waste of code.
>>
>>  Yeah.Other RAP file which references the same code that
>> already in
>>  the first rap should not pull the same code into the final
>> image.
>>
>>
>>  I suppose we could add code to compact the memory and
>> not loaded
>>  the object file and so the overhead is limited to the
>> RAP file.
>>
>>  Sorry. I can not got this. what code should be added to
>> rtl-host?
>>
>>
>> Thinking about this some more I can understand why you did not get
>> it and also your question about host side is a good one. Thinking
>> out loud ...
>>
>> Lets say we have RAP module A and B and both reference
>> 'rtems_rate_monotonic_get___status' which is not resident in the
>>
>> base kernel image. Both RAP modules will get a copy of the object
>> file. We load A first and it's copy is fixed up and
>> 'rtems_rate_monotonic_get___status' is placed in the global symbol
>>
>> table. Any other global symbols in that object file are also placed
>> in the global symbol table. Then we load B and we see
>> 'rtems_rate_monotonic_get___status' is present in the global symbol
>>
>> table. I suspect a duplicate symbol error is raised because we
>> currently do not know if both versions of the symbol match the same
>> code. I suppose a CRC16 could be added to the object file's data and
>> if A and B's versions match we ignore B's and the global symbols can
>> be referenced.
>>
>> If we can determine the same is code is present I suspect removing
>> the unreferenced code in B at load time may be difficult on the
>> target because we have merged the object file's sections together
>> with all the other object files in the RAP file and may not have the
>> required info present to strip it out on target.
>>
>> On the host side is the '--runtime-lib' (-P) option of rtems-ld
>> doing this anyway so why do we need to bother with the above ?
>>
>> Yeah. --runtime-lib handles the common code used by multiple RAP files.
>>
>>
> Great.
>
>
>  We are in need of user documentation for the RTL code.
>>
>> Hah! what kind of doc do you prefer? doxgen doc in patch format or just
>> wiki?  And the documentation is about how to let us

Re: Pass multiple same options to rtems_linkflags

[Chris Johns]

On 29/07/2014 12:15 am, Peng Fan wrote:


 We are in need of user documentation for the RTL code.

Hah! what kind of doc do you prefer? doxgen doc in patch format
or just
wiki?  And the documentation is about how to let user can easily
integrate RTL into his/her application?


Yes, something about how to use the RTL, rtems-ld and what happens
with applications.

ok. I take time to update the wiki with what I have got.



Thanks.





Currently, I am more concerned about another problem which we talked
about when I load python rap and you also talked about with sebh.
lets say that we have a.o b.o c.o and the three .o files references
symbols in libc.a libm.a librtemscpu.a librtemsbsp.a.
Because libc.a libm.a librtemscpu.a librtemsbsp.a is not compile
with
-mlong-calls, so if the rap file is big enough, RTL target may
fail to
load the rap file since reloc entry from libxx.a is near jump,
but dest
symbol is in far away.


I remember but I am not sure of the detail any more. Does the gnu ld
perform some sort of fix up when it does a static link ?

Is this is on the sparc target ?

I only test it on ARM realview qemu platform.  I did not dig into bfd
library, but i think ld can handle it using bfd lib.



Ah ok ARM.




I am hacking it these few days, but still do not have a good idea,
because it is hard to convert reloc entry in libxxx.a from near
reference to far reference as '-mlong-call' does.


The RSB lets you add target specific options. I know it is hack but
it might help. Check rtems/config/4.11/rtems-m32c.__bset for an
example. Maybe you can add the -mlong-call to the sparc build to see
what happens.

using -mlong-call to compile rtems may only make librtemscpu.a and
librtemsbsp.a not use relative reloc. To libc.a and libm.a and libgcc.a,


Using the RSB and the options I suggested rebuilds libc etc. It is still 
a bad hack.



it may not help.Hack rtl-host or ld bfd may help ,but may add arch
sepecific code to rtl-host which is not a good idea.


Yeah it would be nice if rtems-ld could stay neutral but if it cannot 
then that is how it is. If we can keep the platform specific parts 
separate with a suitable class interface it should be ok.


What would rtems-ld have to do on ARM to fix this problem ?


I'll try it on sparc sis.


Great.

Chris
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel