Re: [RFC 0/6] glibc port to ARC architecture

[Joseph Myers]

On Thu, 7 Dec 2017, Vineet Gupta wrote:

> I presume you just want to know 010-glibcs-arc-linux-gnu-check-*.txt after
> running
> 
> scripts/build-many-glibcs.py  glibcs arc-linux-gnu
> 
> FAIL: elf/check-localplt
> Summary of test results:
>   1 FAIL
>1169 PASS
>  15 XFAIL
> 
> And even that failure is weird as
> (1) this is despite my updates to .../arc/localplt.data
> (2) My buildrooot based build reports this test to pass (after my update) but
> still fails in build-many-glibc based build.

The 011 log should show the output of all non-PASS tests, so allowing you 
to identify the problem local PLT entry use (or if applicable, PLT entry 
not used that was expected to be used).  If it's e.g. 
compiler-version-specific and hard to fix for some reason, note how 
entries in localplt.data can use "?" to mark them optional (or can specify 
an alternative relocation, in cases where a function is meant to be 
interposable but may not have a PLT entry).

> Anyhow seems like this should be easy to figure - not mission critical as the
> system running testsuite xcheck is bootstrapped with same ld.so / libc etc.

For build-many-glibcs.py use to detect testsuite regressions for a 
configuration, the baseline required is *no* failures in the parts of the 
testsuite it runs.  If there are any failures at all, that serves to hide 
regressions (additional tests starting to fail, or the testsuite starting 
to fail to build), because it just distinguishes zero / nonzero exit 
status from "make check".

> We are now down to 51 (with github based gcc: more obviously with upstream
> gcc). I think only a very small percentage (~10% guess) would be due to
> missing glibc bits per-se.
> 
> Do you think it would be considered review/merge worthy. I will continue to

I think a plausible state to be merge-ready would be no more than 20 
architecture-specific failures, *with upstream GCC and binutils*.  We've 
had enough problems in the past with glibc ports that turn out to rely on 
non-upstream toolchain pieces that I think you need results with upstream 
tools essentially as good as those with non-upstream before inclusion of 
the port is appropriate.  So you should get whatever GCC fixes are needed 
upstream sooner rather than later.  (That means upstream in GCC mainline 
so GCC 8 is ready for the port.  Release branch backports are at your 
discretion, although it's generally a good idea to make sure the GCC port 
is in a good state in the most recent release branch, where there is any 
GCC release branch supporting the architecture at all.)

That does not mean you should stop work on eliminating failures once down 
to 20, as many ports are rather better than that, just that 20 is more 
comparable with other reasonably well maintained glibc ports.

It's still be appropriate to submit the code for review now, before you're 
down to 20 failures, given that a port is likely to need to go through 
multiple rounds of review anyway before it's ready for inclusion in glibc.

Note that it should be possible for a new port to go into glibc during the 
release freeze period (January), provided it's clear the changes cannot 
affect other ports.  So if you have any fixes to architecture-independent 
parts of glibc that are needed for this port, you should submit them as 
soon as possible, and separately from the main port submission.

> work
> on bringing down failures. Otherwise new changes will mean I keep missing the
> sweeping arch updates / more failures ... I can post the full set of current
> failures if that helps steer decision.

The full list of failures, and whatever analysis you have of their causes 
/ symptoms, should certainly be included in every submission of the port.

-- 
Joseph S. Myers
jos...@codesourcery.com

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Vineet Gupta]

On 11/27/2017 02:16 PM, Joseph Myers wrote:

On Mon, 27 Nov 2017, Vineet Gupta wrote:


Any new port should have support added to build-many-glibcs.py for all
ABIs supported by the port (e.g. both endiannesses, if you support both BE
and LE, and any other ABI variants).


build-many-glibcs.py works for ARC now - after the 2 backports to gcc 7.2


Works with clean compilation test results for the glibc testsuite?


I presume you just want to know 010-glibcs-arc-linux-gnu-check-*.txt after 
running

scripts/build-many-glibcs.py  glibcs arc-linux-gnu

FAIL: elf/check-localplt
Summary of test results:
  1 FAIL
   1169 PASS
 15 XFAIL

And even that failure is weird as
(1) this is despite my updates to .../arc/localplt.data
(2) My buildrooot based build reports this test to pass (after my update) but 
still fails in build-many-glibc based build.


Anyhow seems like this should be easy to figure - not mission critical as the 
system running testsuite xcheck is bootstrapped with same ld.so / libc etc.



You should make sure that produces clean test results for all the
compilation tests, for all those variants.

You should also include results for the full testsuite, including
execution tests (whether testing natively, or cross testing with
test-wrapper set to execute tests for a cross build), in the submission of
the port (and those should be as clean as possible).


ATM we have around 200 failures for upstream tools (likely due to libgcc
unwinder patch not yet merged upstream). And just for data point, with github
based gcc including the non-merged patches that number comes down to ~100.
Bunch of them are in math/doubler and some in backtrace/nptl. Will this be
considered a blocker. I'm almost ready for next round, rebased recently !


You should make sure you regenerate libm-test-ulps for your port (from
scratch, truncate the file and run make regen-ulps).


Thx, that indeed help with quite a few failures.


If you look at

you'll see some known architecture-independent issues that are generic for
certain cases (some cases of cross-testing, particular kernel versions,
etc.).  Beyond anything listed there, I'd say you should have no more than
10-20 FAILs in a well-maintained port, preferably less than that.  100
FAILs indicates there's still some work to do before the port can be
considered to be in a good state.


The 100 were due to lack of c++ support, stale math ulps etc, default sa_restorer 
generated code etc (which libgcc unwinder is choosy about). And then there were 
some genuine fixes such as:


  csu/test-as-const-tcb-offsets
  misc/tst-syscall-list
  dlfcn/tststatic5
  misc/tst-clone3
...

We are now down to 51 (with github based gcc: more obviously with upstream gcc). I 
think only a very small percentage (~10% guess) would be due to missing glibc bits 
per-se.


Do you think it would be considered review/merge worthy. I will continue to work
on bringing down failures. Otherwise new changes will mean I keep missing the
sweeping arch updates / more failures ... I can post the full set of current 
failures if that helps steer decision.


-Vineet

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Joseph Myers]

On Mon, 27 Nov 2017, Vineet Gupta wrote:

> > Any new port should have support added to build-many-glibcs.py for all
> > ABIs supported by the port (e.g. both endiannesses, if you support both BE
> > and LE, and any other ABI variants).
> 
> build-many-glibcs.py works for ARC now - after the 2 backports to gcc 7.2

Works with clean compilation test results for the glibc testsuite?

> > You should make sure that produces clean test results for all the
> > compilation tests, for all those variants.
> > 
> > You should also include results for the full testsuite, including
> > execution tests (whether testing natively, or cross testing with
> > test-wrapper set to execute tests for a cross build), in the submission of
> > the port (and those should be as clean as possible).
> 
> ATM we have around 200 failures for upstream tools (likely due to libgcc
> unwinder patch not yet merged upstream). And just for data point, with github
> based gcc including the non-merged patches that number comes down to ~100.
> Bunch of them are in math/doubler and some in backtrace/nptl. Will this be
> considered a blocker. I'm almost ready for next round, rebased recently !

You should make sure you regenerate libm-test-ulps for your port (from 
scratch, truncate the file and run make regen-ulps).

If you look at 
 
you'll see some known architecture-independent issues that are generic for 
certain cases (some cases of cross-testing, particular kernel versions, 
etc.).  Beyond anything listed there, I'd say you should have no more than 
10-20 FAILs in a well-maintained port, preferably less than that.  100 
FAILs indicates there's still some work to do before the port can be 
considered to be in a good state.

-- 
Joseph S. Myers
jos...@codesourcery.com

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Vineet Gupta]

On 06/27/2017 05:00 AM, Joseph Myers wrote:

On Tue, 27 Jun 2017, Florian Weimer wrote:


On 06/27/2017 10:00 AM, Vineet Gupta wrote:

This is a Request for comments for glibc port to ARC architecture.
http://www.synopsys.com/IP/ProcessorIP/ARCProcessors/Pages/default.aspx

The Linux kernel, uClibc, buildroot have been upstream for quite some time.


Is it possible to build the new port with scripts/build-many-glibcs.py?


More specifically:

Any new port should have support added to build-many-glibcs.py for all
ABIs supported by the port (e.g. both endiannesses, if you support both BE
and LE, and any other ABI variants).


build-many-glibcs.py works for ARC now - after the 2 backports to gcc 7.2


You should make sure that produces clean test results for all the
compilation tests, for all those variants.

You should also include results for the full testsuite, including
execution tests (whether testing natively, or cross testing with
test-wrapper set to execute tests for a cross build), in the submission of
the port (and those should be as clean as possible).


ATM we have around 200 failures for upstream tools (likely due to libgcc unwinder 
patch not yet merged upstream). And just for data point, with github based gcc 
including the non-merged patches that number comes down to ~100. Bunch of them are 
in math/doubler and some in backtrace/nptl. Will this be considered a blocker. I'm 
almost ready for next round, rebased recently !


-Vineet


___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Joseph Myers]

On Mon, 6 Nov 2017, Vineet Gupta wrote:

>   - How do I call it the first time, with my existing version of glibc.
> The checkout process, gets upstream glibc and tries to build that instead.

build-many-glibcs.py  checkout
[replace the src/glibc directory there with a different branch if desired]
build-many-glibcs.py  host-libraries
build-many-glibcs.py  compilers 
build-many-glibcs.py  glibcs 

(It won't actually check that components are checked out from the 
repositories / branches it thinks they are, so replacing by a checkout 
from a different location should work fine.)

It's expected that upstream glibc should work with upstream versions of 
the relevant other toolchain components; we don't want a repeat of the 
NaCl situation, or the situation with MicroBlaze for a long time, where 
required changes either never got upstream or took a very long time to get 
upstream.  It's possible for very new ports that it only works with 
upstream mainline and not the most recent releases, although I'd encourage 
you to act like the RISC-V maintainers and e.g. actively backport 
important fixes to the most recent binutils release branch.

"work" should include clean results from the compilation tests, as run by 
build-many-glibcs.py for the "glibcs" action.

-- 
Joseph S. Myers
jos...@codesourcery.com

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Vineet Gupta]

On 06/27/2017 05:00 AM, Joseph Myers wrote:

On Tue, 27 Jun 2017, Florian Weimer wrote:


On 06/27/2017 10:00 AM, Vineet Gupta wrote:

This is a Request for comments for glibc port to ARC architecture.
http://www.synopsys.com/IP/ProcessorIP/ARCProcessors/Pages/default.aspx

The Linux kernel, uClibc, buildroot have been upstream for quite some time.


Is it possible to build the new port with scripts/build-many-glibcs.py?


More specifically:

Any new port should have support added to build-many-glibcs.py for all
ABIs supported by the port (e.g. both endiannesses, if you support both BE
and LE, and any other ABI variants).


I added the trivial bits to build-many-glibcs.py for supporting ARC, but it seems 
there are some roadblocks.


1. There are still some in-flight patches for gcc/binutils upstreaming. The latest 
working versions are hoisted on on SNPS github portal, but that won't fly I guess. 
Anyhow for now, the upstream versions are likely to suffice for say building part.


2. The immediate (and embarrassing) issue is trying to figure out how to make the 
build-many-glibcs.py script work. A wiki page for new ports would have helped a lot.


  - How do I call it the first time, with my existing version of glibc.
The checkout process, gets upstream glibc and tries to build that instead.

  - So I created a src directory manually, will all other tools and glibc being 
a
symlink to my existing glibc tree and calling it as follows

$ test-glibc/src/glibc>./scripts/build-many-glibcs.py /test-glibc 
compilers  arc-linux-gnu


FAIL: compilers-arc-linux-gnu check-host-libraries
UNRESOLVED: compilers-arc-linux-gnu binutils rm
UNRESOLVED: compilers-arc-linux-gnu binutils mkdir
UNRESOLVED: compilers-arc-linux-gnu binutils configure
UNRESOLVED: compilers-arc-linux-gnu binutils build
...

I'm missing obvious/simple stuff, but this is a new env for me.

-Vineet

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Vineet Gupta]

On 06/27/2017 02:44 PM, Joseph Myers wrote:

I don't see a corporate copyright assignment (or disclaimer plus
individual assignments) from Synopsys in the FSF copyright.list.  The
assignment will be needed before we can consider this port.



Right we missed this ! FWIW we did the agreements for gcc, gdb and binutils and 
will get the paperwork going for glibc as well.


Thx,
-Vineet

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Joseph Myers]

On Tue, 27 Jun 2017, Florian Weimer wrote:

> On 06/27/2017 10:00 AM, Vineet Gupta wrote:
> > This is a Request for comments for glibc port to ARC architecture.
> > http://www.synopsys.com/IP/ProcessorIP/ARCProcessors/Pages/default.aspx
> > 
> > The Linux kernel, uClibc, buildroot have been upstream for quite some time.
> 
> Is it possible to build the new port with scripts/build-many-glibcs.py?

More specifically:

Any new port should have support added to build-many-glibcs.py for all 
ABIs supported by the port (e.g. both endiannesses, if you support both BE 
and LE, and any other ABI variants).

You should make sure that produces clean test results for all the 
compilation tests, for all those variants.

You should also include results for the full testsuite, including 
execution tests (whether testing natively, or cross testing with 
test-wrapper set to execute tests for a cross build), in the submission of 
the port (and those should be as clean as possible).

Any new port should be added to the list in README, and have a NEWS entry 
added.

-- 
Joseph S. Myers
jos...@codesourcery.com

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Joseph Myers]

I don't see a corporate copyright assignment (or disclaimer plus 
individual assignments) from Synopsys in the FSF copyright.list.  The 
assignment will be needed before we can consider this port.

-- 
Joseph S. Myers
jos...@codesourcery.com

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc

Re: [RFC 0/6] glibc port to ARC architecture

[Florian Weimer]

On 06/27/2017 10:00 AM, Vineet Gupta wrote:
> This is a Request for comments for glibc port to ARC architecture.
> http://www.synopsys.com/IP/ProcessorIP/ARCProcessors/Pages/default.aspx
> 
> The Linux kernel, uClibc, buildroot have been upstream for quite some time.

Is it possible to build the new port with scripts/build-many-glibcs.py?

Thanks,
Florian

___
linux-snps-arc mailing list
linux-snps-arc@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-snps-arc