Re: -Werror=implicit-int -Werror=implicit-function-declaration coming to rawhide

[Carlos O'Donell]

On 9/27/23 12:47, Richard W.M. Jones wrote:
> On Wed, Sep 27, 2023 at 11:22:04AM -0500, Ron Olson wrote:
>> This is the first time I’ve heard of buildflags.md; where might I find this
>> file?
> 
> $ ls -l -h /usr/share/doc/redhat-rpm-config/buildflags.md
> -rw-r--r--. 1 root root 28K Feb 28  2023 
> /usr/share/doc/redhat-rpm-config/buildflags.md
> 
> (part of the redhat-rpm-config package)
> 
> I didn't know it existed either :-)

You have 5 years of excellent documentation by Florian and others to catch up 
on! :-)

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 9/27/23 04:43, Peter Robinson wrote:
> On Wed, Sep 27, 2023 at 6:24 AM Remi Collet  wrote:
>>
>> Le 26/09/2023 à 19:32, Carlos O'Donell a écrit :
>>
>>>> In version 8.3 (F40) we'll includes the UTC definition
>>>> in our patch to use system tzdata, UTC being use
>>>> as the fallback value.
>>>
>>> I'm curious; what does this patch look like?
>>
>> (trivial) patch to our system tzdata patch attached
>>
>> In short, if file is missing use bundled content
>>
>> Full patch for PHP 8.3:
>> https://git.remirepo.net/cgit/rpms/scl-php83/php.git/plain/php-8.3.0-systzdata-v24.patch
>>
>> Remi
>>
>>
>> P.S. IMHO, allowing removal of tzdata for PHP doesn't make any sense
>> as most app will fail badly (runtime exception) because of missing
>> TZ when upstream use a bundle copy of the full database, so this can
>> never happen, so this will create another RPM specific problem
> 
> Can't you just explicitly require tzdata?

I agree, if PHP requires tzdata then please add "Requires: tzdata"

With any kind of minimization work where we split up the packages we are going 
to
find implicit dependencies that should be made explicit. It is perfectly OK to
require tzdata from PHP if that's what is needed for your users, and you know
your users best.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: -Werror=implicit-int -Werror=implicit-function-declaration coming to rawhide

[Carlos O'Donell]

On 9/27/23 12:22, Ron Olson wrote:
> This is the first time I’ve heard of buildflags.md; where might I find this 
> file?

It is distributed with redhat-rpm-config as part of the documentation of build 
flags.

You can see the rawhide version here:
https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/rawhide/f/buildflags.md

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On Mon, Sep 25, 2023 at 4:39 AM Vít Ondruch  wrote:
> Dne 22. 09. 23 v 16:01 Zbigniew Jędrzejewski-Szmek napsal(a):
> > On Fri, Sep 22, 2023 at 10:43:05AM +0200, Vít Ondruch wrote:
> >> https://bugzilla.redhat.com/show_bug.cgi?id=2233281#c3
> >>
> >> This probably answers my question. So heads up to others.
> >>
> >> Dne 22. 09. 23 v 10:39 Vít Ondruch napsal(a):
> >>> Was this implemented in past days? I am asking because this FTBFS
> >>> suggest so:
> >>>
> >>> https://koschei.fedoraproject.org/package/rubygem-timecop?collection=f40
> > Yes. The change was done in rawhide a while ago, but it got pushed to F39
> > only recently, see https://bugzilla.redhat.com/show_bug.cgi?id=2233281#c3.
>
> Dealing now with FTBFS rubygem- packages, the change proposal briefly
> mentions: "In June of 2021, we proposed creating a new tzdata
> sub-package that would only provide the UTC timezone.". I assume that
> this have not happened, but I don't remember why and it seems that this
> could be helpful.

In discussions with upstream IANA Tzdata and the Fedora developer
community in 2021, it was concluded that the consensus was to remove
tzdata entirely and fall back to UTC.

The noted problem was that any files provided by tzdata could be an
indication that all of tzdata is installed when that is not the case.

It was clearer to have no files installed and fallback to UTC or all
files installed and provide the full set of functionality.

Today, if a package needs UTC *and* more complex leap second
processing, then they need to install all of tzdata.

Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On Mon, Sep 25, 2023 at 4:52 AM Remi Collet  wrote:
>
> Le 25/09/2023 à 10:38, Vít Ondruch a écrit :
> >
> > Dne 22. 09. 23 v 16:01 Zbigniew Jędrzejewski-Szmek napsal(a):
> >> On Fri, Sep 22, 2023 at 10:43:05AM +0200, Vít Ondruch wrote:
> >>> https://bugzilla.redhat.com/show_bug.cgi?id=2233281#c3
> >>>
> >>> This probably answers my question. So heads up to others.
> >>>
> >>> Dne 22. 09. 23 v 10:39 Vít Ondruch napsal(a):
>  Was this implemented in past days? I am asking because this FTBFS
>  suggest so:
> 
>  https://koschei.fedoraproject.org/package/rubygem-timecop?collection=f40
> >> Yes. The change was done in rawhide a while ago, but it got pushed to F39
> >> only recently, see
> >> https://bugzilla.redhat.com/show_bug.cgi?id=2233281#c3.
> >
> >
> > Dealing now with FTBFS rubygem- packages, the change proposal briefly
> > mentions: "In June of 2021, we proposed creating a new tzdata
> > sub-package that would only provide the UTC timezone.". I assume that
> > this have not happened, but I don't remember why and it seems that this
> > could be helpful.
>
> We have the same issue with PHP and lot of recent FTBFS
>
> timezone is really mandatory for PHP

If they are mandatory for building then you may need a "BuildRequires:
tzdata" in your package now, as you may no longer have it included
transitively.

If your APIs need tzdata at runtime then you may need a "Requires:
tzdata" if the APIs cannot operate in UTC-only mode with tzdata
removed (minimized container).

> In version 8.3 (F40) we'll includes the UTC definition
> in our patch to use system tzdata, UTC being use
> as the fallback value.

I'm curious; what does this patch look like?

Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/28/23 19:54, Michael Catanzaro wrote:
> Because Recommends and Supplements are installed by default, we need
> to be careful and use them sparingly, only when it really makes sense
> for some package to be pulled in for almost all users of another
> package. Thus far, Fedora and RHEL have done a good job of this,
> primarily because we were very late to the weak dep party and have
> had much more time to learn best practices and much less time to
> screw up. This is in contrast to some other distros where it's common
> for users to disable weak deps to avoid "bloat." We don't ever want
> our Recommends/Supplements to be considered bloat. (That's what
> Suggests/Enhances is for. :) And since they're not bloat, they should
> be respected when building most non-minimal images.

I agree.

Do you consider the removal of tzdata to be a valuable use of Recommends?

It is my opinion that glibc should Recommends: tzdata because it frees up an 
optional
~8MiB of zoneinfo that will never be used by UTC-only microservice workloads. 
It is
a further refinement of having really small containers with LANG=C.UTF-8/C and 
TZ=UTC.

Some spec files will need BuildRequires: tzdata, because their %check and 
testsuite
run needs it. In addition to this we should continue to adopt Fedora CI and 
test the
results as required in an installed scenario (with or without tzdata as your 
package
might need to test).

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/29/23 00:44, Miro Hrončok wrote:
> On 29. 06. 23 0:31, Carlos O'Donell wrote:
>> Can we allow users to create a minimal installation by hand, while still 
>> complying
>> with PEP-615 for default installs?
>>
>> The size savings for a minimal container that is UTC-only would be quite 
>> valuable
>> for Fedora minimal containers.
> Yes, but see the rest of my email.
> 

Just for clarity, and 3-way-communication:

(a) You are concerned about the UTC case failing today.

(b) You would like to see an upstream patch to python to detect missing tzdata 
and report something like:

ZoneInfoNotInstalledError: 'No time zone information installed on the 
system, you can only use UTC'

(c) You would like to ensure UTC works even without tzdata installed.

(d) You don't want to carry a downstream patch, but you would be OK with a 
backported upstream patch?

Does that match your expectations?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Is there a chance to phase out `/lib64` directory?

[Carlos O'Donell]

On 6/27/23 20:21, Kevin Kofler via devel wrote:
> Practical cross-compilation to a completely different architecture needs 
> sysroots anyway. That way, one can also easily target a different 
> distribution on a different (or even the same) architecture, not just Fedora 
> on a different architecture.

+100

And assembling those sysroots is not straight forward.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Red Hat & Fedora -- largely stepping out of this ecosystem

[Carlos O'Donell]

On 6/26/23 18:47, Jeff Law wrote:
> What Red Hat has done may be technically legal and perhaps good for
> its business.  However, to me it's ethically unconscionable.   Those
> who know me know I'm not an zealot, but I do have a baseline set of
> ethical values and Red Hat crossed that line.

Why is it ethically unconscionable? There is a lot of confusion around
what has happened and why. What you are saying, and what actually happened
don't line up in my mind :-)

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/28/23 18:24, Fabio Valentini wrote:
> On Thu, Jun 29, 2023 at 12:15 AM Carlos O'Donell 
> wrote:
>> 
>> On 6/26/23 16:12, Fabio Valentini wrote:
>>> So all this considered, I'm not sure whether this change is
>>> actually worth it, if tzdata databases of some form will likely
>>> be pulled into installs anyway.
>> 
>> Quoting the "Weak Dependencies Policy":
>> 
>> "Weak dependencies allow smaller minimal installations while
>> keeping the default installation feature rich."
>> 
>> The change is worth it for minimizing container runtimes based on
>> Fedora.
>> 
>> Just for clarity, a default install will always have tzdata.
>> 
>>> I'd rather have *one* tzdata that's up-to-date and used by
>>> everything (for example, so Python and Ruby programs can actually
>>> agree what time it is).
>> I agree strongly with this statement.
>> 
>> I have worked with Patsy over the years to remove as many bundled
>> copies of out-of-date tzdata as we can find :-)
>> 
>> It is important for consistency that all language runtimes have the
>> same concept of time.
>> 
>> This change request does not change that.
> 
> Thanks for the clarification, though this is only partially
> reassuring: I'm not sure how this accounts for the fact that there
> are some situations in which weak dependencies are *not installed at
> all*. Most notably, they are not installed into build environments
> *at all* (i.e. mock sets `install_weak_deps=0`).

Yes, this is a general issue with all weak dependencies.

This is similar to the "minimal buildroot" conversation, whose conclusion was
that we should not dictate what should be in the minimal buildroot but have
the packages express the dependencies as required for their build and check
phases.

See:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#buildrequires
~~~
You MAY assume that enough of an environment exists for RPM to function, 
to build packages and execute basic shell scripts, but you SHOULD NOT 
assume any other packages are present as RPM dependencies and anything
brought into the buildroot by the build system can change over time.
~~~
https://pagure.io/packaging-committee/issue/497

We have two options:

- glibc-devel 'Requires: tzdata' (close to the current behaviour)
- All packages that neeed tzdata in rpm %check tests add 'BuildRequires: tzdata'

If a testsuite runs in %check and needs tzdata it seems to me that it is 
valuable
to express this as 'BuildRequires: tzdata'. Therefore I lean towards the latter
solution of having packages that truly need tzdata for testing all of their
code to 'BuildRequires: tzdata', but only if actually required.
 
> I'm not sure how the process that builds ISOs and container images 
> handle this, but I assume they set this option as well. So we'd need
> to be careful not to remove "hard" dependencies on tzdata and replace
> them with lots of "weak" dependencies all over the place - just to
> find out that it ends up missing from important places because they
> disable weak dependencies.

Building ISOs and container images must install Recommends: by default otherwise
the Weak Dependency Policy would not work in Fedora.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/26/23 14:14, Peter Robinson wrote:
> On Mon, Jun 26, 2023 at 7:10 PM Miro Hrončok  wrote:
>> *PEP 615 – Support for the IANA Time Zone Database in the Standard Library* 
>> says:
>>
>>
>> """
>> Python distributors are encouraged to ensure that time zone data is installed
> 
> The wording of "encouraged to ensure" doesn't sound like a hard
> requirement to me based on a lot of specs I've dealt with, but it
> depends a bit on how the specific spec defines "encouraged".
> 
>> alongside Python whenever possible (e.g. by declaring tzdata as a dependency
>> for the python package).
>> """
>>
>> from https://peps.python.org/pep-0615/#system-time-zone-information
>>
>>
>> By changing the Requires to Recommends, we would diverge from upstream
>> recommendation.

I agree with Peter. The "Recommends:" will ensure tzdata is installed by 
default.

This work lines up exactly with the Fedora Weak Dependencies Policy:
"Weak dependencies allow smaller minimal installations while keeping the default
 installation feature rich."

Can we allow users to create a minimal installation by hand, while still 
complying
with PEP-615 for default installs?

The size savings for a minimal container that is UTC-only would be quite 
valuable
for Fedora minimal containers.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/26/23 14:40, Vít Ondruch wrote:
> Yep, this is the case for rubygem-tzinfo. It would deserve recommends
> at minimum, because in theory, the tzdata can be suplied by
> tzinfo-data gem instead.

Thank you for adding the `Recommends: tzdata`!

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/26/23 16:12, Fabio Valentini wrote:
> So all this considered, I'm not sure whether this change is actually
> worth it, if tzdata databases of some form will likely be pulled into
> installs anyway.

Quoting the "Weak Dependencies Policy":

"Weak dependencies allow smaller minimal installations while keeping the 
default 
 installation feature rich."

The change is worth it for minimizing container runtimes based on Fedora.

Just for clarity, a default install will always have tzdata.

> I'd rather have *one* tzdata that's up-to-date and used by everything
> (for example, so Python and Ruby programs can actually agree what time
> it is).
I agree strongly with this statement.

I have worked with Patsy over the years to remove as many bundled copies of 
out-of-date
tzdata as we can find :-)

It is important for consistency that all language runtimes have the same 
concept of time.

This change request does not change that.

With a weak dependency on tzdata the default installs will still have it, but 
someone
making a specifically UTC-only container should be able to remove it.

Python, and other language runtimes should be capable of getting to a point 
where they
operate correctly with only C.UTF-8 (no language packs) and UTC (no tzdata).

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/27/23 05:38, Jiri Vanek wrote:
> JDK will behave similarly. We ave (small) advantage that we have also
> in-jdk-bundled tzdata.  However fallback in case of removed system
> tzdata is not automatic, and requires human touch. Long ago we have a
> patch in jdk which looked to system tzdata - if they were present,
> they were used.  If not, the bundled copy was used.  Also user could
> set up on startup which to use. But it had not prooved itself, as it
> was casue of weird missonfigurations.

Please note that we are only talking about tzdata, not tzdata-java.

We do not intend to allow the removal of tzdata-java because OpenJDK depends 
upon it.

When you say "in-jdk-bundled tzdata" do you mean something other than 
tzdata-java?

> If this proposal will come live, we may introduce this patch again, or
> leave it as it is now - on human touch.

Could you please expand a bit more on this topic?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Allow Removal of tzdata (System-Wide)

[Carlos O'Donell]

On 6/28/23 03:44, Miroslav Lichvar wrote:
> On Mon, Jun 26, 2023 at 04:54:00PM +0100, Aoife Moloney wrote:
>> * Other developers: Some packages need to change their spec files from
>> `Requires: tzdata` to `Recommends: tzdata`. It would be beneficial if
>> all packages switched in this way, but it is not required. Supporting
>> optional tzdata installation for as many workloads as possible allows
>> those workloads to minimize their container image size.
>> List of packages which need to be changed:
>> * glibc (glibc-common)
> 
> There are packages that rely on the glibc->glibc-common->tzdata
> dependency. The one I know is chrony, which in the default
> configuration needs the "right/UTC" timezone to get the TAI-UTC
> offset. I suspect there are other packages that will need to add
> Recommends or Requires.
 
Thanks, filed against chrony to fix that.

chrony: Uses right/UTC and needs Requires: tzdata
https://bugzilla.redhat.com/show_bug.cgi?id=2218368

My opinion is that this all part of the work required to make the distribution
more flexible for deploying as the basis for containerized workloads.

If you know of any other such missing dependencies we should file bugs and
get them fixed.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Web Assembly on Fedora: interested in a Fedora SIG to work on this?

[Carlos O'Donell]

On 11/18/22 13:41, Michael Dawson wrote:
> As Web Assembly (WASM) gains momentum we’d like to create a SIG as a place to 
> collaborate to ensure that Fedora is a great platform to both build and run 
> WASM workloads. This includes looking at the toolchains needed to build WASM 
> as a target and the runtimes needed to run WASM. It will provide a place to 
> bring together efforts across different ecosystems (nodejs, rust, compiler 
> toolchains, etc.) as well as a place where people can provide self-help when 
> building and running WASM workloads.
> 
> If you are interested in a WASM Sig please let us know.

I'm interested!

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: hardened memory allocate port to linux-fedora system for secutiry

[Carlos O'Donell]

On 8/26/22 12:22, Daniel Micay via devel wrote:
> Also, you hardened_malloc doesn't use a thread cache for security
> reasons. It invalidates many of the security properties. If you compare
> to glibc malloc in the light configuration with tcache disabled in glibc
> malloc it will compare well, and hardened_malloc can scale better when
> given enough arenas. If you want to make the substantial security
> sacrifices required for a traditional thread cache, then I don't think
> hardened_malloc makes sense, which is why it doesn't include the option
> to do thread caching even though it'd be easy to implement. It may one
> day include the option to do thread batched allocation, but it isn't
> feasible to do it for deallocation without losing a ton of the strong
> security properties.

I'm an upstream glibc developer, but I've tried to remove my bias here and 
present
the facts as they are for the existing heap-based allocator that is in use by 
the
distributions today and why it's hard to change.

(1) Pick your own allocator vs. use the default.

We allow any end user to make those choices by interposing the final allocator 
with
an allocator of their choice depending on specific workload criteria. This means
that distributions don't have a strong incentive to change system allocators 
unless
they are making a strategic change in their core values or vision for the 
distribution
(like Graphene OS makes for security).

At the ELF level we make sure that we can interpose a new allocator, and we work
carefully to ensure that newer features at the compiler level can be supported
incrementally (_FORTIFIY_SOURCE=3 and __builtin_dynamic_object_size) by newer
allocators.

In summary: If the "good enough" allocator doesn't meet your requirements, then
you can use one of the alternatives.

(2) Switching the default vs. improving the default.

It is arguably lower TCO for all distributions using glibc to improve glibc's
malloc. Some improvements can't be made, but some buy enough benefit that there
is no strong reason to change allocators.

For example:
- jemalloc/tcmalloc used a fast per-thread cache.
  - glibc implemented fast per-thread caching in 2.26 (2017) (DJ Delorie's work)

- Chromium started using safe-linking pointer hardening.
  - glibc implemented safe-linking pointer hardening for fastbins and tcache 
(2020) (Eyal Itkin's work)

Next steps for glibc's malloc is probably:

- Improve internal fragmentation [1]
- Round-robin arena assignment with uniform arena assignment as a goal.
- Provide a packed arena for sub 16-byte sized allocations to improve 
utilization.
 - We have seen some C++ workloads/frameworks that create trillions of 13-byte 
objects.

(3) Requirements vs. change.

While Facebook/BSD (jemalloc), Google (tcmalloc), Microsoft (mimalloc) have very
good allocators, issues seen with those allocators can be more difficult to
correct because of the impact those changes have on wider workloads beyond
distribution workloads.

For example if Graphene OS, with it's own goals, and Fedora with it's own goals
had a conflict of interest for the direction of the allocator e.g. cost vs.
security, what kind of choice would the hardened_allocator maintainers make?

Upstream glibc has largely been aligned with traditional distribution
requirements for a long time, and continues to be aligned with the notion
of a "general purpose" distribution via the contributors and deep network
of developers in the distributions:
https://sourceware.org/glibc/wiki/MAINTAINERS#Distribution_Maintainers

---

The combination of (1), (2) and (3) mean that for general purpose
distributions the choice of staying with glibc's malloc means having
an ecosystem of distributions that are using the same allocator and
benefit from wide application testing and development and support
when required.

It would be easier to approach glibc upstream and convince them that the
default allocator in glibc should be replaced with hardened_alloc or
jemalloc or tcmalloc or mimalloc...

-- 
Cheers,
Carlos.

[1] 
https://patchwork.sourceware.org/project/glibc/patch/xn4jz19fts@greed.delorie.com/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F37 proposal: Add -fno-omit-frame-pointer to default compilation flags (System-Wide Change proposal)

[Carlos O'Donell]

On 6/30/22 04:54, Kevin Kofler via devel wrote:
> I am still strongly opposed to degrading performance and size for all users 
> just to help the handful users of poorly-designed profiling tools.

I agree.

My career experience has been that the performance impact of having an extra 
register
for compiler scheduling and to reduce register pressure has real and tangible 
benefit.

I have had to use frame pointers, but only for deeply embedded projects where 
the cost
tradeoffs are different and a smaller constrained unwinder was needed.

I would not recommend the use of -fno-omit-frame-pointer in Fedora.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Need help with LTO running out of memory

[Carlos O'Donell]

On 5/17/22 21:39, Orion Poplawski wrote:
> libfabric 1.15.1 builds on x86_64 are failing because the final LTO link 
> seems to consume all available memory:
...
> gcc: fatal error: Killed signal terminated program lto1
> compilation terminated.
> lto-wrapper: fatal error: gcc returned 1 exit status
> compilation terminated.
> /usr/bin/ld: error: lto-wrapper failed
> 
> For now I'll just disable LTO on x86_64, but should I file a bug somewhere?

Yes please. File the bug against Fedora gcc. Failure in lto1 is a problem 
reading the generated
LTO and processing it for optimization.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F36 Change: GNU Toolchain Update (gcc 12, glibc 2.35) (late System-Wide Change proposal)

[Carlos O'Donell]

On 1/11/22 13:00, Steve Grubb wrote:
> Hello,
> 
> On Wednesday, January 5, 2022 5:05:26 PM EST Ben Cotton wrote:
>> https://fedoraproject.org/wiki/Changes/GNUToolchainF36
>>
>> == Summary ==
>> Update the Fedora 36 GNU Toolchain to gcc 12 and glibc 2.35.
>>
>> The gcc 12 is currently under development and will be included in
>> Fedora 36 upon release. The glibc 2.35 change will be tracked in this
>> top-level GNU Toolchain system-wide update.
> 
> Reading through the GCC 12 changes, there is a significant new feature to GCC 
> that would appear to be useful for security. There is a new:
> 
> -ftrivial-auto-var-init=zero
> 
> flag that initializes all stack variables to zero. Zero being a nice safe 
> value that makes programs crash instead of being exploitable.
> 
> Are there plans to enable this flag so that all applications, but more 
> importantly the kernel, are hardened against uninitialized stack variables? 
> This is one of the major classes of security bugs that could potentially be 
> eliminated during this mass rebuild.

There are currently no plans that I am aware of that involve turning on
'-ftrivial-auto-var-init=zero' in the short term for Fedora. CC'ing Jakub
and Marek to comment.

It is something that should be discussed, turned on in Rawhide first,
and likely via redhat-rpm-config default flags first, and then we should
fix any fallout.

I'd only be comfortable if we did it early and worked through the consequences.
So it could be something to discuss for F37.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Heads up: libffi 3.4 rebuild in rawhide today

[Carlos O'Donell]

On 1/11/22 13:45, Vít Ondruch wrote:
> Thread 1 "ruby" received signal SIGABRT, Aborted.
> 0x778a764c in __pthread_kill_implementation () from /lib64/libc.so.6
> Missing separate debuginfos, use: dnf debuginfo-install 
> glibc-2.34.9000-36.fc36.x86_64 gmp-6.2.1-1.fc36.x86_64 
> libxcrypt-4.4.27-1.fc36.x86_64 zlib-1.2.11-30.fc35.x86_64
> (gdb) bt
> #0  0x778a764c in __pthread_kill_implementation () from 
> /lib64/libc.so.6
> #1  0x7785a656 in raise () from /lib64/libc.so.6
> #2  0x77844833 in abort () from /lib64/libc.so.6
> #3  0x74d0a5b8 in dlfree (mem=0x77bf1010) at 
> ../src/dlmalloc.c:4350

This is libffi's internal allocator detecting an inconsistency.

> #4  0x74d190b1 in dealloc (ptr=0x558c1c00) at 
> /builddir/build/BUILD/ruby-3.1.0/ext/fiddle/closure.c:32

This is fiddle calling ffi_closure_free() because USE_FFI_CLOSURE_ALLOC is 
non-zero.

The original closure was allocated in allocate() in fiddle.

What happened to the closure between allocation and free?

Does the memory location change?

Does something corrupt the closure?

> #5  0x77cb7801 in run_final (zombie=140737300557440, 
> objspace=0xd800) at /builddir/build/BUILD/ruby-3.1.0/gc.c:4011
> #6  finalize_list (objspace=objspace@entry=0xd800, 
> zombie=140737300557440) at /builddir/build/BUILD/ruby-3.1.0/gc.c:4030
> #7  0x77cb80cc in rb_objspace_call_finalizer 
> (objspace=0xd800) at /builddir/build/BUILD/ruby-3.1.0/gc.c:4194
> #8  0x77ca56eb in rb_ec_finalize (ec=0xdd70) at 
> /builddir/build/BUILD/ruby-3.1.0/eval.c:164
> #9  rb_ec_cleanup (ec=ec@entry=0xdd70, ex0=) at 
> /builddir/build/BUILD/ruby-3.1.0/eval.c:256
> #10 0x77ca5c14 in ruby_run_node (n=0x77699660) at 
> /builddir/build/BUILD/ruby-3.1.0/eval.c:321
> #11 0x518f in main (argc=, argv=) 
> at ./main.c:47
> (gdb) f 3
> #3  0x74d0a5b8 in dlfree (mem=0x77bf1010) at 
> ../src/dlmalloc.c:4350
> 4350      USAGE_ERROR_ACTION(fm, p);

We only get here when the incoming pointer is invalid.

> (gdb) l
> 4345      check_free_chunk(fm, p);
> 4346      goto postaction;
> 4347        }
> 4348      }
> 4349        erroraction:
> 4350      USAGE_ERROR_ACTION(fm, p);
> 4351        postaction:
> 4352      POSTACTION(fm);
> 4353        }
> 4354      }

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Heads up: libffi 3.4 rebuild in rawhide today

[Carlos O'Donell]

On 1/10/22 14:04, Miro Hrončok wrote:
> On 10. 01. 22 18:14, Miro Hrončok wrote:
>> On 08. 01. 22 18:09, Carlos O'Donell wrote:
>>> On 1/8/22 04:37, Miro Hrončok wrote:
>>>> Hello packagers,
>>>>
>>>> I intent to rebuild the following packages with libffi 3.4 in Rawhide...
>>
>> -
>>
>> Not yet rebuilt packages:
>>
>> $ repoquery -q --repo=koji --whatrequires libffi3.1 --source | pkgname
>> gambas3
>> hadolint
>> jffi
>> llvm
>> llvm10
>> llvm11
>> llvm12
>> llvm9.0
>> python2.7
>> python3.6
>> python3.7
>> ruby
>> thunderbird
>> xs
>>
>>
>> All are known. jffi seems to be the only libffi-related failure.
> 
> OK, ruby also seems related:
> 
> https://koschei.fedoraproject.org/package/ruby
 
Agreed.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Heads up: libffi 3.4 rebuild in rawhide today

[Carlos O'Donell]

On 1/8/22 04:37, Miro Hrončok wrote:
> Hello packagers,
> 
> I intent to rebuild the following packages with libffi 3.4 in Rawhide side 
> tag f36-build-side-49314 today.

Thank you for helping with the rebuilds!
 
> The previous version remains available as libffi13.1, so failures to build 
> will not result in uninstallable packages.
> 
> You can inspect some known failures:
> 
> https://copr.fedorainfracloud.org/coprs/churchyard/libffi-3.4/package/cjs/
> https://copr.fedorainfracloud.org/coprs/churchyard/libffi-3.4/package/gjs/
> both "killed by signal 11 SIGSEGV"

Right, this is because there is an ordering dependency between 
gobject-introspection and
cjs/gjs. You need the introspection library rebuilt first with libffi 3.4+ and 
then build
the javascript packages, that way they both have the same library and can pass 
objects
back and forth for introspection.
 
> https://copr.fedorainfracloud.org/coprs/churchyard/libffi-3.4/package/hadolint/
> Not all RPM dependencies satisfied

Agreed.

DEBUG util.py:444:  No matching package to install: 'ghc-colourista-prof'
DEBUG util.py:444:  No matching package to install: 'ghc-ilist-prof'
DEBUG util.py:444:  No matching package to install: 'ghc-spdx-prof'
DEBUG util.py:444:  No matching package to install: 'ghc-timerep-prof'
DEBUG util.py:444:  Not all dependencies satisfied
DEBUG util.py:444:  Error: Some packages could not be found.

Rawhide did build successfully on 2021-11-30, but that was while ago and the 
deps have issues now.

> https://copr.fedorainfracloud.org/coprs/churchyard/libffi-3.4/package/jffi/
> make: *** No rule to make target '-L/usr/lib64/../lib64', needed by 
> '/builddir/build/BUILD/jffi-jffi-1.3.4/build/jni/jffi/Array.o'.  Stop.

I don't know why this one fails. Passed in c9s with earlier jffi and libffi 3.4.

Rawhide did build successfully on 2021-08-22, but that was a while ago.

> https://copr.fedorainfracloud.org/coprs/churchyard/libffi-3.4/package/ruby/
> Segmentation fault
> FAIL 1/1489 tests failed

This failed in the same place in two different builds in the test_ractor.rb 
(Ruby Ractor)
test case, and it crashes in 'ractor_select()' within 'rb_vm_exec()'.

This is odd that it should fail with the libffi update since this the failure 
is in the
Ruby Ractor test, which I wouldn't expect to use any of the FFI APIs. It has 
failed
twice though in the same place.

I didn't see this in c9s. The last built ruby in c9s was built by me and it is 
3.0.2-155,
where test_ractor.rb passes just fine built with libffi 3.4.

The ruby-mri binary has no deep DT_NEEDED dependencies which should need libffi 
or other
libraries to be built in a particular order, but with dlopen you can get odd 
ordering
issues that are only resolved after the SONAME bump is complete and rebuilds 
completed
across dependent libraries.

Rawhide did build successfully on 2021-12-10.

> https://copr.fedorainfracloud.org/coprs/churchyard/libffi-3.4/package/thunderbird/
> collect2: error: ld returned 1 exit status

The logs don't contain any more information. This is a static linker failure 
when building
libxul.so.

Rawhide did build successfully on 2021-12-15.
 
> https://copr.fedorainfracloud.org/coprs/churchyard/libffi-3.4/package/xs/
> Run-time dependency ffi found: NO (tried pkgconfig and cmake)

Run-time dependency ffi found: NO (tried pkgconfig and cmake)
Library ffi found: YES
^^
Run-time dependency gc found: NO (tried pkgconfig and cmake)
Library gc found: YES
Library gccpp found: YES
Run-time dependency readline found: YES 8.1
Program touch found: YES (/usr/bin/touch)
Program ../generators/buildinfo.sh found: NO
meson.build:24:2: ERROR: Program '../generators/buildinfo.sh' not found or not 
executable
  
^^^
A full log can be found at 
/builddir/build/BUILD/XS-789540c5f208b8e8f07fc81c3bec3d0ee47c6dea/build/meson-logs/meson-log.txt

xs has been FTBS since July 2021:
https://koji.fedoraproject.org/koji/buildinfo?buildID=1805437

> 
> TIP: If your package is present in c9s, consider looking how it was fixed 
> there.
> 
> 
> The rest of the packages I'll rebuild (passed on x86_64 in copr):
> 
> Agda
> alex
> bench
> brainfuck
> bustle
> cab
> cabal-install
> cabal-rpm
> cpphs
> darcs
> dfuzzer
> dhall
> dhall-json
> dl-fedora
> ecl
> fbrnch
> firefox
> gambas3
> gforth
> ghc
> ghc-aeson-pretty
> ghc-cabal-helper
> ghc-clientsession
> ghc-criterion
> ghc-DAV
> ghc-doctest
> ghc-hakyll
> ghc-HaXml
> ghc-hgettext
> ghc-highlighting-kate
> ghc-hjsmin
> ghc-hspec-discover
> ghc-cheapskate
> ghcid
> ghc-libffi
> ghc-pretty-show
> ghc-servant-server
> ghc-vty
> ghc-wai-app-static
> ghc-wai-websockets
> ghc8.10
> ghc9.0
> ghc9.2
> git-annex
> gitit
> git-repair
> glib2
> gnustep-base
> gobject-introspection
> gtk2hs-buildtools
> guile
> guile22
> guile30
> happy
> haskell-platform
> hedgewars
> hledger
> hledger-ui
> hledger-web
> hlint
> hscolour
> idris
> jna
> libomp

Re: Raising the attachment size limit in bugzilla?

[Carlos O'Donell]

On 12/14/21 12:37, Daniel P. Berrangé wrote:
> A sosreport contains a tonne of useful info, but for any single bug
> the vast majority is irrelevant. So it is much harder to argue that
> requesting this sos report info is proportionate for solving bugs
> from Fedora users, especially when attachments default to public
> and never expire.

I'm happy if Fedora SOS reports default to *less* information, and perhaps have
no logs, and if that gets us under the 19.5MiB attachment limit then I'm fine.

I'm looking for an *easy* way to get /proc/cpuinfo and other system information
from a developer since that helps me as a glibc developer track down bugs.
The existence of sos makes this process easier (no matter if it gathers too much
information).

I have not seen any Fedora-specific customization for sos (the package e.g. 
sos.spec)
but it should be possible to do that.

I could file a bug asking for a Fedora-specific customization to default to
collecting less information?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Raising the attachment size limit in bugzilla?

[Carlos O'Donell]

On 12/14/21 10:16, Robbie Harwood wrote:
> Carlos O'Donell  writes:
> 
>> - Life-cycle management (delete attachments).
> 
> Please don't delete attachments.  It severely reduces the usefulness of
> keeping old bugzillas around - if we're going to do that, we might as
> well delete the old bugzilla entries as well, and I don't think anyone
> wants that.

I noted "life-cycle management" specifically so we could have a discussion 
about the
topic. Choosing one way or the other has costs and consequences. Without data 
from
bugzilla about the total size, growth rate of attachments, and cost of storage, 
it's
hard to decide on a real life-cycle policy.

To say "we must keep it all" needs some very specific qualification, because 
often
the older the bugzilla the less useful it is because it no longer matches 
existing
in-use code. Yes, it is nice for archaeology, but is it sufficiently nice that 
we
would prioritize it *over* the needs of Fedora users today to upload SOS 
reports?

Two positions could arise, given a fixed budget for storage:
(a) We keep attachments forever, but users can't upload SOS reports.
(b) We keep attachments for a reasonable amount of time, and users can upload 
SOS reports.

If I understand your data retention policy correctly it looks like this:
- Maximize usefulness.
  - Priority is to existing and new <19.5MiB attachments?
  - What about the priority to users and their ability to upload SOS reports?
- Consequence: Keep data forever and pay for that storage?

Do you have any thoughts on archiving attachments older than a certain age into
some kind of slow access / low cost / cold storage via a bugzilla URL 
attachment?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: pipewire memory usage

[Carlos O'Donell]

On 12/14/21 07:08, Dominique Martinet wrote:
> I've double-checked with traces in load_spa_handle/unref_handle and it
> is all free()d as soon as the client disconnects, so there's no reason
> the memory would still be used... And I think we're just looking at some
> malloc optimisation not releasing the memory.
> 
> To confirm, I've tried starting pipewire-pulse with jemalloc loaded,
> LD_PRELOAD=/usr/lib64/libjemalloc.so , and interestingly after the 100
> clients exit the memory stays at ~3-400MB but as soon as single new
> client connects it jumps back down to 20MB, so that seems to confirm it.
> (with tcmalloc it stays all the way up at 700+MB...)

 
> So I guess we're just chasing after artifacts from the allocator, and
> it'll be hard to tell which it is when I happen to see pipewire-pulse
> with high memory later on...

It can be difficult to tell the difference between:
(a) allocator caching
(b) application usage

To help with we developed some additional tracing utilities:
https://pagure.io/glibc-malloc-trace-utils

The idea was to get a full API trace of malloc family calls and then play them 
back
in a simulator to evaluate the heap/arena usage when threads were involved.

Knowing the exact API calls lets you determine if you have (a), where the API 
calls
show a small usage but in reality RSS is higher, or (b) where the API calls 
show there
are some unmatched free()s and the usage is growing.

It seems like you used jemalloc and then found that memory usage stays low?

If that is the case it may be userspace caching from the allocator.

jemalloc is particularly lean with a time-decay thread that frees back to the OS
in order to reduce memory usage down to a fixed percentage. The consequence of
this is that you get latency on the allocation side, and the application has to
take this into account.

> From what I can see the big allocations are (didn't look at lifetime of each
> alloc):
>  - load_spa_handle for audioconvert/libspa-audioconvert allocs 3.7MB
>  - pw_proxy_new allocates 590k
>  - reply_create_playback_stream allocates 4MB
>  - spa_buffer_alloc_array allocates 1MB from negotiate_buffers
>  - spa_buffer_alloc_array allocates 256K x2 + 128K
>from negotiate_link_buffers

On a 64-bit system the maximum dynamic allocation size is 32MiB.

As you call malloc with ever larger values the dynamic scaling will scale up to
at most 32MiB (half of a 64MiB heap). So it is possible that all of these 
allocations
are placed on the mmap/sbrk'd heaps and stay there for future usage until freed 
back.

Could you try running with this env var:

GLIBC_TUNABLES=glibc.malloc.mmap_threshold=131072

Note: See `info libc tunables`.

> maybe some of these buffers sticking around for the duration of the
> connection could be pooled and shared?
 
They are pooled and shared if they are cached by the system memory allocator.

All of tcmalloc, jemalloc, and glibc malloc attempt to cache the userspace 
requests
with different algorithms that match given workloads.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Raising the attachment size limit in bugzilla?

[Carlos O'Donell]

The Fedora SOS reports are ~30MiB today, and this exceeds the
Bugzilla attachment limit of 19.5MiB.

Do we have the option to raise the attachment size to something
that could accommodate the average SOS report limit for Fedora
uses? I've had users report SOS tarballs that are ~60MiB in size
which would mean we need a ~100MiB attachment limit (5x what we
have today).

This is a difficult problem to solve because we need:
- Storage (real costs).
- Life-cycle management (delete attachments).

Comments?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Fedora 35 Mass Rebuild

[Carlos O'Donell]

On 7/22/21 12:13 PM, Tomas Hrcka wrote:
> They haven't finished building, so if we cancel both of these, then it
> would work:
> 
>> https://koji.fedoraproject.org/koji/taskinfo?taskID=72392389
>> https://koji.fedoraproject.org/koji/taskinfo?taskID=72392356
>>
> 
> 
> I have canceled both tasks.

Perfect. I've blocked them from building again. Thanks.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Fedora 35 Mass Rebuild

[Carlos O'Donell]

On 7/20/21 12:02 PM, Tomas Hrcka wrote:
> Per the Fedora 35 schedule[1] we will start a mass rebuild for Fedora 35
> on Jul 21st, 2021. We will run a mass rebuild for Fedora 35 for the
> changes listed in:
> 
> https://pagure.io/releng/issues?status=Open&tags=mass+rebuild

F35 rebase to libffi 3.4
https://pagure.io/releng/issue/10213

The intent here was to *delay* the libffi 3.4.2 / libffi3.1 rebase until
after the mass rebuild and then do a targetted rebuild.

Unfortunately I failed to push my %build exit 1 changes (since changes
were live in Fedora Rawhide with changes, built in a side-tag, 
and well tested, but not built into the f35 tag).

Therefore the mass rebuild *includes* the libffi 3.4.2 transition, and
the libffi 3.1 inclusion.

They haven't finished building, so if we cancel both of these, then it would 
work:
https://koji.fedoraproject.org/koji/taskinfo?taskID=72392389
https://koji.fedoraproject.org/koji/taskinfo?taskID=72392356

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: tzdata-minimal (Self-Contained Change proposal)

[Carlos O'Donell]

On 7/15/21 6:34 AM, Zbigniew Jędrzejewski-Szmek wrote:
> On Tue, Jul 06, 2021 at 01:20:47PM -0400, Ben Cotton wrote:
>> https://fedoraproject.org/wiki/Changes/tzdata-minimal
>>
>> == Summary ==
>> Split the tzdata package into two parts - tzdata and tzdata-minimal.
>> tzdata will require tzdata-minimal.  tzdata-minimal provides the
>> minimal files needed to support UTC on containers.
>>
>> == Owner ==
>> * Name: Patsy Griffin (Franklin)
>> * Email: pa...@redhat.com
>>
>>
>> == Detailed Description ==
>> This is the first step towards providing support for a minimal, UTC
>> only, version of tzdata for containers.  The tzdata-minimal package
>> will be a stand-alone, UTC only, subset of tzdata. The tzdata package
>> will require tzdata-minimal.
>>
>> With this framework in place, other packages can develop code to
>> detect a minimal tzdata installation.  These packages will also need
>> to provide appropriate messages when users request timezone
>> information not available when only tzdata-minimal is installed.
> 
> In general, I like the idea of making it easier to not install tzdata.
> For many machines it's completely unnecessary, and 5MB is enough to
> make this worthwhile.

Thanks for supporting the idea.

> I don't know enough about all the consumers to comment on the details.
> FWIW, systemd-timedated already has a reasonable message that encompasses
> both possible errors:
> 
>   $ timedatectl set-timezone Europe/Warsaw2
>   Failed to set time zone: Invalid or not installed time zone 'Europe/Warsaw2'
> 
> But I think it'd be useful to use a different message if we can
> distinguish the two cases. I'd be happy to take a patch...

Yes, you could say "invalid" (zone table files installed) if the zone is not
in one of the tables, and "invalid or not installed" if there are no files
(no tzdata installed).

We are still in the process of checking with upstream tzdata if they are
comfortable with, and will support *not* installing some of the zones, and
document that upstream.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: libffi 3.4 (late System-Wide Change proposal)

[Carlos O'Donell]

On 7/15/21 6:52 AM, Zbigniew Jędrzejewski-Szmek wrote:
> On Wed, Jul 14, 2021 at 02:28:50PM -0400, Ben Cotton wrote:
>> https://fedoraproject.org/wiki/Changes/LIBFFI34
>>
>> == Summary ==
>> Rebase libffi in Fedora 34 from libffi 3.1 to libffi 3.4 (released
>> June 28 2021), and provide a libffi3.1 compatibility package to handle
>> the library SONAME transition.
> 
> The usual: please don't say "rebase" in public-facing documents.
> It's packaging jargon that is not clear to users.

Switched to "Update" since that's what we're doing.

Thanks for the feedback! I've been filing system-wide change requests for
years with glibc, and I was never quite sure of the exact audience that
might be reading them.

I'll try to make the text more "user friendly" in the future. I certainly
appreciate the feedback.

> Looks all reasonable… With the compat package, this should be very
> smooth.

I agree, the compat package is required, it really makes the whole
transition very smooth.
 
>> == Release Notes ==
>>
>> libffi does not public formal release notes.  A change list can be
>> found in libffi's git history:
>> https://github.com/libffi/libffi/commits/master
> 
> Those sentences are slightly garbled too.

Upstream is providing release notes.

I've updated them to list the important notes for Fedora users.

Many will be happy about the RISC-V support.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: libffi 3.4 (late System-Wide Change proposal)

[Carlos O'Donell]

On 7/14/21 3:00 PM, Michael Catanzaro wrote:
> On Wed, Jul 14 2021 at 02:28:50 PM -0400, Ben Cotton  
> wrote:
>> Rebase libffi in Fedora 34
> 
> Is this a typo? F34 or F35?

Sorry. Fixed. Typo.

We've been carrying this change proposal since F34 as we work with
upstream to release libffi 3.4.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: libffi 3.4 (late System-Wide Change proposal)

[Carlos O'Donell]

On 7/14/21 2:33 PM, Samuel Sieb wrote:
> On 7/14/21 11:28 AM, Ben Cotton wrote:
>> == Upgrade/compatibility impact ==
>> Packages built on the previous version of libffi will have an
>> auto-requires on the old SONAME and will cause dnf to install
>> libffi3.1 (compat package with runtime). When packages are rebuilt
>> against libffi 3.4 they will automatically switch to using the new
>> SONAME.
>>
>> libffi's latest static trampolines feature has been disabled because
>> it has an impact on
> 
> Looks like this section got cut off.

Sorry. Updated.

Thee static templates impact gobject-introspection and ghc.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: tzdata-minimal (Self-Contained Change proposal)

[Carlos O'Donell]

On 7/13/21 8:12 AM, Petr Viktorin wrote:
> On 12. 07. 21 18:31, Carlos O'Donell wrote:
>> If you can minimally provide the tables of possible zones, and
>> provide an easy way to detect a zone is missing, then the APIs can
>> determine: "Yes you could do that, but your system is missing the
>> data." vs. "That is an invalid zone." Which is a useful
>> distinction.
> 
> Sure, we can provide such an API, but that alone won't make consumers
> use it. Especially if it's specific to Fedora (and Clear Linux --
> AFAIK, people expect missing bits and pieces are there.)
> 
> I think the best place to discuss and document this would be the tz.
> If their README (or just a mail thread) said the data can be split
> this way, and hinted at what should be done when it is, we wouldn't
> need this conversation. Without that, I don't think it's OK to allow
> Fedora systems with only tzdata-minimal.

That's a fair point. We'll get clarification from upstream and Paul Eggert.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: tzdata-minimal (Self-Contained Change proposal)

[Carlos O'Donell]

On 7/13/21 11:17 AM, Petr Viktorin wrote:
> On 13. 07. 21 16:31, Carlos O'Donell wrote:
>> It's possible to remove all tzdata.
>>
>> However, without *some* data it is not easy to distinguish between these
>> two scenarios if you want to offer a different error message:
>>
>> - Is this zone provided and correct but missing? e.g. exists but is not 
>> installed.
>>    - zone table exists, lists the zone, but the zone is missing.
>>
>> - Is this zone not correct? e.g. doesn't exist in the current version.
>>    - zone table doesn't list the zone.
> 
> True. But I don't see a use case where this would be important
> enough: when is an error message like "tzdata is not installed, so
> the zone can't be used *or* checked" not enough?

It may be enough.

If it turns out to be not enough would we be ready to reconsider the
packaging decision?

> The problem with making it possible to distinguish between these two
> scenarios is that *every* consumer of tzdata must now distinguish
> between them. If they assume tzdata is never split (as many do,
> AFAIK), their error messages will be wrong. That's not every consumer
> we distribute in a RPM (we could handle that I guess), but also any
> third-party one people will build/install on their systems.

Understood.

(1) Missing files vs. No /usr/share/zoneinfo.

All existing third-party software must handle zone name changes today,
so there must be some way to handle the error that a given zone is
missing (changed names).

Let me talk a bit about the new C++ time zone API I have been looking
at with Jonathan Wakely.

Errors based on this will be likely correct e.g. get_tz_dir() from the
currently proposed C++ API for this (see (2)):

 353 CONSTDATA auto tz_dir_default = "/usr/share/zoneinfo";
 354 CONSTDATA auto tz_dir_buildroot = "/usr/share/zoneinfo/uclibc";
 355 
 356 // Check special path which is valid for buildroot with uclibc builds
 357 if(stat(tz_dir_buildroot, &sb) == 0 && S_ISDIR(sb.st_mode))
 358 return tz_dir_buildroot;
 359 else if(stat(tz_dir_default, &sb) == 0 && S_ISDIR(sb.st_mode))
 360 return tz_dir_default;
 361 else
 362 throw runtime_error("discover_tz_dir failed to find zoneinfo\n");

Having tzdata uninstalled will throw the generic error as on line 362.

But having tzdata-minimal will instead throw a specific error from line 3624:

3624 throw std::runtime_error(std::string(tz_name) + " not found in 
timezone database");

This probably supports having tzdata removed entirely, since the latter
error makes it seems as-if the zone name is wrong (it's not, it's just
not installed).

Notes:
https://github.com/HowardHinnant/date

(2) Non-rpm packaging.

We have C++ users using Howard Hinnart's 'date' package that implements
 header and it can process complete IANA tzdb files.

Currently Howard's API is being proposed for inclusion 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0355r7.html

So the error returned in the runtime_error can have an explanatory string,
so it could be tailored to a system with a partially installed tzdata-minimal,
but it could likewise detect the entire tzdata is uninstalled (no files)
and just say that (as you note, and the code supports today).

So there may not be a strong case for making the distinction between the
two error modes.

>>> We could theoretically patch Python to give reasonable error
>>> messages. But since the consumer of the tz data (the zoneinfo module)
>>> was only added in Python 3.9 (last year), existing applications
>>> mainly use third-party modules instead of the standard library. I
>>> assume that like Python, these modules expect tzinfo to either be
>>> missing entirely or be all there. And I expect this is the case for
>>> more than just Python modules.
>>   Over time some zones become deprecated and invalid names.
>>
>> One must already handle zone name changes, so if the code can handle
>> name deprecation then it will report the same error for missing zoneinfo. >
>>> Is it reasonable for glibc to hardcode the +0 fallback timezone,
>>> rather than needing the zoneinfo file for it? If so, we could remove
>>> tzdata from minimal containers entirely. Or is that too naive?
>>
>> It is not naive.
>>
>> glibc already falls back to UTC with no data present (and we need to cleanup
>> what we print).
>>
>> The question is what kind of errors we want to be able to express to users.
>>  
>>>>> == Benefit to Fedora ==
>>>>> This change will reduce the size of base container installations.
>>>>>
>>

Re: F35 Change: tzdata-minimal (Self-Contained Change proposal)

[Carlos O'Donell]

On 7/12/21 12:16 PM, Petr Viktorin wrote:
> On 06. 07. 21 20:38, David Cantrell wrote:
>> On Tue, Jul 06, 2021 at 01:20:47PM -0400, Ben Cotton wrote:
>>> https://fedoraproject.org/wiki/Changes/tzdata-minimal
>>>
>>> == Summary ==
>>> Split the tzdata package into two parts - tzdata and tzdata-minimal.
>>> tzdata will require tzdata-minimal.  tzdata-minimal provides the
>>> minimal files needed to support UTC on containers.
>>>
>>> == Owner ==
>>> * Name: Patsy Griffin (Franklin)
>>> * Email: pa...@redhat.com
>>>
>>>
>>> == Detailed Description ==
>>> This is the first step towards providing support for a minimal, UTC
>>> only, version of tzdata for containers.  The tzdata-minimal package
>>> will be a stand-alone, UTC only, subset of tzdata. The tzdata package
>>> will require tzdata-minimal.
>>>
>>> With this framework in place, other packages can develop code to
>>> detect a minimal tzdata installation.  These packages will also need
>>> to provide appropriate messages when users request timezone
>>> information not available when only tzdata-minimal is installed.
>>>
>>> == Feedback ==
>>> We have had requests for this functionality in order to support
>>> minimal container installations.  Currently some container kickstart
>>> installations already ad hoc remove most of the timezone information
>>> provided by tzdata, leaving only UTC support available.  This change
>>> provides a formal method of providing this support.
>>>
>>> Both the glibc and python teams are aware of this proposed change.
>>> This change does not currently require changes in their code.  The
>>> goal is for those packages that currently require tzdata as part of
>>> their build or install, move towards recommending tzdata instead.
> 

> Python should work well without *any* tzdata installed. But having
> only a small subset of timezones would result in issues unique to
> Fedora-based systems (assuming those are the only systems that split
> tzdata).

It's possible to remove all tzdata.

However, without *some* data it is not easy to distinguish between these
two scenarios if you want to offer a different error message:

- Is this zone provided and correct but missing? e.g. exists but is not 
installed.
  - zone table exists, lists the zone, but the zone is missing.

- Is this zone not correct? e.g. doesn't exist in the current version.
  - zone table doesn't list the zone.
 
> We could theoretically patch Python to give reasonable error
> messages. But since the consumer of the tz data (the zoneinfo module)
> was only added in Python 3.9 (last year), existing applications
> mainly use third-party modules instead of the standard library. I
> assume that like Python, these modules expect tzinfo to either be
> missing entirely or be all there. And I expect this is the case for
> more than just Python modules.
 
Over time some zones become deprecated and invalid names.

One must already handle zone name changes, so if the code can handle
name deprecation then it will report the same error for missing zoneinfo.
 
> Is it reasonable for glibc to hardcode the +0 fallback timezone,
> rather than needing the zoneinfo file for it? If so, we could remove
> tzdata from minimal containers entirely. Or is that too naive?

It is not naive.

glibc already falls back to UTC with no data present (and we need to cleanup
what we print).

The question is what kind of errors we want to be able to express to users.
 
>>> == Benefit to Fedora ==
>>> This change will reduce the size of base container installations.
>>>
>>> == Scope ==
>>> * Proposal owners: Implement the proposal.
>>> * Other developers: Developers need to ensure that their packages
>>> continue to build and install with the new split tzdata/tzdata-minimal
>>> package changes.
>>>
>>> * Release engineering: No coordination required with Release Engineering.
>>> * Policies and guidelines: The policies and guidelines do not need to
>>> be updated.
>>> * Trademark approval: N/A (not needed for this Change)
>>> * Alignment with Objectives: N/A
>>>
>>> == Upgrade/compatibility impact ==
>>> The only visible change will be a new package tzdata-minimal required by 
>>> tzdata.
>>>
>>>
>>> == How To Test ==
>>> Run a dnf upgrade of tzdata and observe that tzdata-minimal is now
>>> also installed as a dependency.
>>>
>>>
>>> == User Experience ==
>>> Users will see that new updates to tzdata include a new package
>>> dependency on tzdata-minimal.
>>>
>>>
>>> == Dependencies ==
>>> This change does not require or depend on changes to other packages.
>>> However, we hope that dependent packages will work towards
>>> recommending tzdata for builds and installs rather than requiring it.
>>>
>>>
>>> == Contingency Plan ==
>>> * Contingency mechanism: If we are unable to complete this feature by
>>> the final development freeze, we will revert to the shipped
>>> configuration.
>>> * Contingency deadline: 100% Code complete deadline
>>> * Blocks release? No
>>>
>>> == Documentation ==
>>> No documentati

Re: F35 Change: tzdata-minimal (Self-Contained Change proposal)

[Carlos O'Donell]

On 7/6/21 2:38 PM, David Cantrell wrote:
> On Tue, Jul 06, 2021 at 01:20:47PM -0400, Ben Cotton wrote:
>> https://fedoraproject.org/wiki/Changes/tzdata-minimal
>>
>> == Summary ==
>> Split the tzdata package into two parts - tzdata and tzdata-minimal.
>> tzdata will require tzdata-minimal.  tzdata-minimal provides the
>> minimal files needed to support UTC on containers.
>>
>> == Owner ==
>> * Name: Patsy Griffin (Franklin)
>> * Email: pa...@redhat.com
>>
>>
>> == Detailed Description ==
>> This is the first step towards providing support for a minimal, UTC
>> only, version of tzdata for containers.  The tzdata-minimal package
>> will be a stand-alone, UTC only, subset of tzdata. The tzdata package
>> will require tzdata-minimal.
>>
>> With this framework in place, other packages can develop code to
>> detect a minimal tzdata installation.  These packages will also need
>> to provide appropriate messages when users request timezone
>> information not available when only tzdata-minimal is installed.
>>
>> == Feedback ==
>> We have had requests for this functionality in order to support
>> minimal container installations.  Currently some container kickstart
>> installations already ad hoc remove most of the timezone information
>> provided by tzdata, leaving only UTC support available.  This change
>> provides a formal method of providing this support.
>>
>> Both the glibc and python teams are aware of this proposed change.
>> This change does not currently require changes in their code.  The
>> goal is for those packages that currently require tzdata as part of
>> their build or install, move towards recommending tzdata instead.
>>
>> == Benefit to Fedora ==
>> This change will reduce the size of base container installations.
>>
>> == Scope ==
>> * Proposal owners: Implement the proposal.
>> * Other developers: Developers need to ensure that their packages
>> continue to build and install with the new split tzdata/tzdata-minimal
>> package changes.
>>
>> * Release engineering: No coordination required with Release Engineering.
>> * Policies and guidelines: The policies and guidelines do not need to
>> be updated.
>> * Trademark approval: N/A (not needed for this Change)
>> * Alignment with Objectives: N/A
>>
>> == Upgrade/compatibility impact ==
>> The only visible change will be a new package tzdata-minimal required by 
>> tzdata.
>>
>>
>> == How To Test ==
>> Run a dnf upgrade of tzdata and observe that tzdata-minimal is now
>> also installed as a dependency.
>>
>>
>> == User Experience ==
>> Users will see that new updates to tzdata include a new package
>> dependency on tzdata-minimal.
>>
>>
>> == Dependencies ==
>> This change does not require or depend on changes to other packages.
>> However, we hope that dependent packages will work towards
>> recommending tzdata for builds and installs rather than requiring it.
>>
>>
>> == Contingency Plan ==
>> * Contingency mechanism: If we are unable to complete this feature by
>> the final development freeze, we will revert to the shipped
>> configuration.
>> * Contingency deadline: 100% Code complete deadline
>> * Blocks release? No
>>
>> == Documentation ==
>> No documentation changes are needed at this time.
>>
>>
>> == Release Notes ==
>> The tzdata package is now divided into a UTC only package,
>> tzdata-minimal, and tzdata.
> 
> What is supposed to be in tzdata-minimal?  Is it
> /usr/share/zoneinfo/UTC or that and more?

A little more is provided.

You need some of the name tables for certain use cases.

This is based on Intel's clear linux work in splitting the data.
 
> Forcibly removing tzdata on a fresh Fedora VM that I just set up has
> the system fall back to UTC, as expected.  On this incredibly small
> install, tzdata is required glibc-common, python3-libs, and
> python3-dateutil.  The last one is reasonable, but for all of them I
> ask if tzdata is actually a hard dependency or if it can become a weak
> dependency and this change proposal could become "make tzdata
> something easily removable" rather than creating more tzdata packages.

If you can minimally provide the tables of possible zones, and provide
an easy way to detect a zone is missing, then the APIs can determine:
"Yes you could do that, but your system is missing the data."
vs.
"That is an invalid zone."
Which is a useful distinction.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: F35 Change: GNU Toolchain update (gcc 11, glibc 2.34, binutils 2.37, gdb 10.2) (System-Wide Change proposal)

[Carlos O'Donell]

On Tue, Jun 29, 2021 at 2:14 PM Ben Cotton  wrote:
>
> On Tue, Jun 29, 2021 at 2:07 PM Ben Cotton  wrote:
> >
> > == Contingency Plan ==
> > * Contingency mechanism: If glibc 2.34 provides too disruptive to
> > compiling the distribution we could revert to 2.33, but given that
> > Rawhide has started tracking glibc 2.34, no show-stopper problems are
> > expected.  At this point, we can still revert to upstream version 2.33
> > if insurmountable problems appear, but to do so may require a mass
> > rebuild to remove new symbols from the ABI/API.
> > * Contingency deadline: Upstream glibc ABI freeze deadline of 2021-07-01.
> >
> I notice that the listed contingency deadline is two days from now, so
> it will have passed before this even makes it to FESCo for vote. Is
> that date correct?

The date is flexible. But undoing the ABI changes requires a mass rebuild.

Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it: 
https://pagure.io/fedora-infrastructure

Re: Fedora Release page on wiki not created?

[Carlos O'Donell]

On 11/20/19 2:22 PM, Ben Cotton wrote:
> 
> 
> On Wed, Nov 20, 2019, 19:00 Carlos O'Donell  <mailto:car...@redhat.com>> wrote:
> 
> The Fedora change template here: 
> https://fedoraproject.org/wiki/Changes/EmptyTemplate
> 
> Says you should reference the "Targeted release" in the status.
> 
> For F31 and F32 it doesn't appear as if the pages were created: 
> https://fedoraproject.org/wiki/Releases/31/ 
> https://fedoraproject.org/wiki/Releases/32/
> 
> Is there someone I should ping about this?
> 
> 
> You just did. :-) Much of that content is in the early stages of
> being moved to docs.fedoraproject.org
> <http://docs.fedoraproject.org>. The existence of that page isn't
> important for the changes process, there just needs to be a number so
> I know what version the proposal is for.
 
If it's a link from the change request template we should put *something*
on that page as placeholder so people don't get a missing link if they
happen to click on it? :-)

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Fedora Release page on wiki not created?

[Carlos O'Donell]

The Fedora change template here:
https://fedoraproject.org/wiki/Changes/EmptyTemplate

Says you should reference the "Targeted release" in the status.

For F31 and F32 it doesn't appear as if the pages were created:
https://fedoraproject.org/wiki/Releases/31/
https://fedoraproject.org/wiki/Releases/32/

Is there someone I should ping about this?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: clang and Fedora's default C/CXX flags

[Carlos O'Donell]

On 1/3/19 2:49 PM, Michael Cronenworth wrote:
> Part of Fedora's default C/CXX flags include -fstack-clash-protection
> but clang does not support this flag and has until a few weeks ago[1]
> silently ignored the flag.
> 
> What are clang apps who use Fedora's default flags supposed to do?
> Are there clang default flag macros?
> 
> Thanks, Michael
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1658311

You should reach out to Tom or Serge for comments :-)

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Delaying the ppc64le transition to 128-bit IEEE long double.

[Carlos O'Donell]

Just a heads up that the ppc64le ABI transition from IBM long double to 
128-bit IEEE long double has been delayed.

I've removed the following from F30:
https://fedoraproject.org/wiki/Changes/PPC64LE_Float128_Transition
(moved it back to ChangePageIncomplete).

I've also indicated this is the case here:
https://pagure.io/releng/issue/7907

If there is anything else I need to do to remove the system-wide
change request from F30, please tell me.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Multilib inconsistencies between fedora/updates/updates-testing composes

[Carlos O'Donell]

On 12/14/18 1:50 PM, Kevin Fenzi wrote:
> On 12/14/18 4:06 AM, Florian Weimer wrote:
>> * Florian Weimer:
>>
>>> We have seen reports that glibc-headers.i686 comes and goes from the
>>> x86_64 updates compose.  Previously, we have seen this only for the
>>> updates-testing compose: 
>>>
>>> This leads to a very bad update experience.  Users file bugs against the
>>> glibc package, but I don't think we can do anything on our side, at
>>> least not until we know what the actual compose bug is and what triggers
>>> it.
>>
>> We really need to fix this issue, it breaks updates for many users.
>>
>> Any suggestion how we can move this forward?
> 
> I had another question for lsedlar, but the prepopulate might work. That
> means we bloat our updates/updates-testing repos by all the .i686 stuff
> that was in the GA repo, but at least it means it will always be there.

Yes please. Consistency is the key here. Either all gone, or all there.
 
> This also would mean it would require changing this list if someone
> changed a package in a update such that it was no longer multilib, but
> hopefully that doesn't happen too often.

This is so rare I've never observed it ;-)

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Bodhi updates permanently locked in PENDING

[Carlos O'Donell]

On 12/14/18 2:08 PM, Kevin Fenzi wrote:
> On 12/14/18 4:38 AM, Florian Weimer wrote:
>> I need to make changes to these updates:
>>
>>   https://bodhi.fedoraproject.org/updates/FEDORA-2018-2efb53dc71
>>   https://bodhi.fedoraproject.org/updates/FEDORA-2018-0e5b278265
>>
>> They have been locked in the pending state for hours.  How can I get
>> them unstuck?
> 
> They are stuck because the push yesterday at 00:00 failed due to the
> switch outage still happening. We are working on getting them to finish
> and push out. As soon as the compose they are in completes, they will be
> unlocked.

Thanks for the update Kevin. I'll keep monitoring this. If there is anything
I can do from our side please tell me.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: Guideline change: glibc malloc as the C/C++/Rust allocator

[Carlos O'Donell]

On 07/27/2018 06:43 AM, Florian Weimer wrote:
> On 07/26/2018 06:03 PM, Jason L Tibbitts III wrote:
>>> "FW" == Florian Weimer  writes:
>> 
>> FW> I would like to request a change of the Packaging Guidelines, 
>> FW> advising packagers not to interpose malloc.
>> 
>> How strong of a restriction are you looking for?  This sort of
>> feels like something which would at the strongest be a "SHOULD NOT"
>> but maybe you're looking for an absolute ban.
> 
> An absolute ban strikes me as overly broad.  However, especially for
> shared objects (such as libruby), the implications should be clear:
> it is easy to LD_PRELOAD a different malloc, but if the library is
> linked against glibc malloc, jemalloc, and then you want to preload
> tcmalloc because it helps with your workload, you might run into
> problems (and if it's just excessive use of TLS data, most of it
> unused).

Agreed.

>> Some maintainers may find it difficult to comply with an absolute
>> ban if the relevant software doesn't make it easy to change the
>> allocator.
> 
> Sure, this is why I mentioned APR pools and Boehm GC—if the APIs a
> different, then we obviously can't require porting to system malloc.
> 
> If upstream bundles a custom malloc, especially one of the well-known
> ones separately packaged in Fedora, some work is required anyway
> because the bundled version cannot be installed in a system-wide
> location, and RPM provides for it must be suppressed.  (See the
> upstream varnish packages for an example of an accidental system-wide
> jemalloc installation.)

Right, if the relevant software doesn't make it easy to change the
allocator then that's OK, but what we ship must be made safe to ship.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/UV5XGF3HM3BPCEARZEYKNIG4OGIECXBO/

Re: Guideline change: glibc malloc as the C/C++/Rust allocator

[Carlos O'Donell]

On 07/26/2018 12:45 PM, R P Herrold wrote:
> On Thu, 26 Jul 2018, Florian Weimer wrote:
> 
>> I would like to request a change of the Packaging Guidelines, advising
>> packagers not to interpose malloc.
> 
> The use here of 'interpose' is unclear to me -- are you saying 
> 'substitute a different' ?
> 
>> The reasons are:
>>
>> * We have resources to support glibc malloc, but not for other mallocs.
>> * Other mallocs  do not follow ABI and provide insufficient alignment.
>> * Choosing a malloc is workload-dependent and forcing a non-default
>>   malloc takes options away from system administrators.
> 
> Could you please mention a couple of bugs where this is shown?
Sure, issues like bug 1430223 are CLOSED/WONTFIX.

Bug 1430223 - In some conditions, tcmalloc memalign will segfault
https://bugzilla.redhat.com/show_bug.cgi?id=1430223

Do we have anyone with allocator experience beyond the Fedora glibc team?

I think a key point here is to reduce the number of allocators being
used by the distribution so we can keep the quality high and help
our users when they have problems.

Granted some people will need to use jemalloc because upstream links
against it directly, or is deeply integrated with it. I don't think
we should block that. We should however, avoid it where possible, and
standardize on an allocator that works well by default across a lot of
workloads, and let the system admins / DevOps people choose allocators
in the light of feedback from performance on production workloads
(not chosen by us, package managers, or upstream).

Allocator choice is always tightly coupled to workload.

Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/XHADC733NTQXRVSHNAYP2OLB7XLJDPRP/

Re: [HEADS UP] gcc/gcc-c++ removal from buildroot and more

[Carlos O'Donell]

On 07/09/2018 07:03 PM, Igor Gnatenko wrote:
> today we finally dropped gcc and gcc-c++ from the buildroot
> .
> This made 12 packages go away along with 134MB installed size. This
> means that you need to add gcc/gcc-c++ in the BuildRequires
> (guidelines stated this for few years but not many were following
> them).

Great job!

In November 2015 (almost 3 years now) we added this requirement to
the packaging guidelines specifically for C and C++ :-)

"Packaging:C and C++"
https://fedoraproject.org/wiki/Packaging:C_and_C%2B%2B

Cheers,
CArlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/DU4I4FPN4WAOB4PMK6LBOMHHP6WTHBBQ/

Re: Guideline change: glibc malloc as the C/C++/Rust allocator

[Carlos O'Donell]

On 07/26/2018 12:24 PM, Vít Ondruch wrote:
> 
> 
> Dne 26.7.2018 v 18:03 Jason L Tibbitts III napsal(a):
>>> "FW" == Florian Weimer  writes:
>> FW> I would like to request a change of the Packaging Guidelines,
>> FW> advising packagers not to interpose malloc.
>>
>> How strong of a restriction are you looking for?  This sort of feels
>> like something which would at the strongest be a "SHOULD NOT" but maybe
>> you're looking for an absolute ban.
>>
>> Some maintainers may find it difficult to comply with an absolute ban if
>> the relevant software doesn't make it easy to change the allocator.  And
>> some upstreams may have strong preferences for one allocator over
>> another (which I imagine would often be linked to performance).  Will
>> assistance be offered to modify affected packages?  What about
>> contacting upstreams?
>>
>> How many packages would need to change to meet this guideline?
> 
> Just FTR, there is ongoing discussion about changing default to jemalloc
> in Ruby:
> 
> https://bugs.ruby-lang.org/issues/14718
> 
> Wouldn't it be better if glibc maintainers joined such discussions to
> improve glibc malloc implementation?

This is an orthogonal problem. I've responded in the Ruby ticket.

Improving glibc's malloc will take time, and we have already started.

We tackled performance for qemu and DJ Delorie implemented lockless thread
caching to reduce the fast path just like tcmalloc and jemalloc.

The next step is to tackle RSS and it is poorly understand and non-trivial.

Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/HNCIDBGWPRTWHVHSMPHLJCJ5IV4A54DV/

FYI: Potential delay in toolchain readiness for 64-bit POWER LE transition to 128-bit IEEEE long double.

[Carlos O'Donell]

FYI.

As part of the glibc 2.28 development upstream plans to change the 
ppc64le default ABI for long double from "IBM long double" to 
"IEEE 128-bit long double", this change is documented here:
https://fedoraproject.org/wiki/Changes/PPC64LE_Float128_Transition

We have a potential problem here in that the required changes may 
not be ready for 2018-07-11 mass rebuild per the current
Fedora 29 schedule: https://fedoraproject.org/wiki/Releases/29/Schedule

I will keep the following tickets updated:
(Releng) https://pagure.io/releng/issue/7475
(FESco) https://pagure.io/fesco/issue/1925

Upstream may delay the ABI freeze by two weeks:
https://www.sourceware.org/ml/libc-alpha/2018-06/msg00818.html
which means an  optimal mass rebuild might be a week later on
2018-07-18.

To be clear I am not requesting any change at this point. I am 
only providing a heads-up that there is a potential for a delay 
here or if it comes to it a complete cancellation of the float128 
transition for ppc64le (which we wouldn't like to see, but there 
are deadlines).

Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/BOHQ6Z4YYFQ2VWKEA6MR7G4WOIAIWOSH/

Re: glibc, riscv64, multilib, /lib64 etc

[Carlos O'Donell]

On 02/08/2018 07:14 AM, Richard W.M. Jones wrote:
> Or when building glibc, use:
> 
>   ./configure --libdir=/usr/lib64 libc_cv_slibdir=/usr/lib64 
> libc_cv_rtlddir=/usr/lib64
> 
> which seems to be sufficent to override the default path choices,
> although maybe not completely.

This would be a mistake IMO, since it creates a variant ABI.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: glibc, riscv64, multilib, /lib64 etc

[Carlos O'Donell]

On 02/08/2018 06:51 AM, david.abdurachma...@gmail.com wrote:
>> We could do a downstream patch.
> 
> This would be a minimal patch based on quick look into glibc code. We
> could force it to act as, e.g AArch64. I worry, that this would be
> custom from what is expected in RISC-V software ecosystem. If we stay
> with /lib64/lp64d/ then we probably need to do more fixing in packages,
> because I doubt everything is using %{_libdir}.
> 
> Alternative being discussed right now at #fedora-riscv is to use
> symlinks, e.g. /usr/lib64/lp64d -> /usr/lib64 like we have /lib64 ->
> /usr/lib64
> 
> If we can solve this issue with symlinks, then I would vote for that.
> Then we don't need to do a lot of fixing and we sorta have what's
> expected by glibc, or/and in general RISC-V software ecosystem.

I see two issues at hand:

(a) RISC-V software ecosystem compatibility.

(b) Technical implications and ABI compatibility.

In regards to (a), I agree completely with David, that deviating
from the RISC-V software ecosystem is going to cause adoption problems.
We should avoid changing what upstream has chosen. Symlinks should
solve the issue, and if they do not then that's a bug in the dynamic
loader and someone should immediately file the bug and the Fedora
glibc team will fix it.

In regards to (b), there are two very important distinctions to make:

(1) What is the path to the dynamic loader?

(2) What is the path to shared libraries?

Firstly, (1) is an ABI atrifact that all 3rd party libraries will
have encoded into the binaries PT_INTERP program header entry, and
for binaries compiled with an upstream the official path is listed
here:

https://sourceware.org/glibc/wiki/ABIList#RISC-V

Surprisingly it is /lib. So this means that rtlddir is /lib, and
the dynamic loader is uniquely placed in /lib *even though* it
is a 64-bit dynamic loader.

We *must* *not* deviate from this in Fedora, and this path must
exist to the loader, but it can be a symlink.

Secondly, (2) is all the libraries, and these we have much more
flexibility, we can really put them wherever we want, the DSO's
DT_NEEDED entries are SONAMES which are searched by our own loader
and since we control those search paths we could make things
work like we expect them to with all libraries in /lib64.

However, the upstream community has chosen /lib64/lp64 as the
location of the shared library directory e.g. slibdir, and that
means 3rd party binaries could have hard coded dlopen paths that
try to open specific libraries e.g. dlopen /lib64/lp64/libfoo.so.0
and if we don't at least symlink that path it will fail. Likewise
3rd party binaries may also use DT_RPATH and DT_RUNPATH to specify
specific search paths which may be based on /lib64/lp64 e.g.
/lib64/lp64/mozilla/plugins.

In summary:

- We must minimally provide a symlink for /lib/ld-linux-riscv64-lp64d.so.1 
- We must minimally provide a symlink for /lib64/lp64d
- As Fedora glibc team lead I'm OK with symlinks.

I'm happy to see both of these symlinks in glibc.spec point to
wherever we as a community decide we want to see things.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Rawhide make segfaults.

[Carlos O'Donell]

At present I can't build glibc in rawhide because make segfaults:

+ make -j4 -O -r 'CFLAGS=-mtune=generic -g -O2  -fstack-clash-protection'
make -r PARALLELMFLAGS="" -C .. objdir=`pwd` all
make: *** [Makefile:9: all] Segmentation fault (core dumped)
error: Bad exit status from /var/tmp/rpm-tmp.g0o5R3 (%build)

cvc4 shows the same problem:

Making all in src
/bin/sh: line 12: 16161 Segmentation fault  (core dumped) make $local_target

We should tag in the last known good make.

I've run out of time to debug this but I wanted to pass on my notes.

(1) Run the testsuite.

If the package you rebuild has a testsuite please make sure you run it
and review the results.

The make testsuite was not run, there is a driver problem with it.

Sadly, the make testsuite hasn't been run since F26:
https://kojipkgs.fedoraproject.org//packages/make/4.2.1/2.fc26/data/logs/x86_64/build.log
where it passed with no failures.

We should strive hard to make sure that testsuite runs every time,
and that the results are 100% pass.

(2) make segfaults here.

#0  0x557b1ad1 in ?? ()
#1  0x76fd4c7b in glob_in_dir () from /lib64/libc.so.6
#2  0x76fd5b9f in glob@@GLIBC_2.27 () from /lib64/libc.so.6
#3  0x5557551c in parse_file_seq ()
#4  0x555673f7 in func_wildcard ()
#5  0x55569501 in handle_function ()
#6  0x55562cfb in variable_expand_string ()
#7  0x55563833 in allocated_variable_expand_for_file ()
#8  0x555671cf in func_foreach ()
#9  0x55569501 in handle_function ()
#10 0x55562cfb in variable_expand_string ()
#11 0x555773c7 in eval ()
#12 0x555775f1 in eval_makefile ()
#13 0x555767ce in eval ()
#14 0x555775f1 in eval_makefile ()
#15 0x55577a38 in read_all_makefiles ()
#16 0xe375 in main ()

You have to set follow-fork-mode child, and set detach-on-fork off
to catch the faulting child process in gdb (and flip between processes
to run them in sequence).

(3) Building make-4.2.1-5 in F27 works.

A `fedpkg local` build of make make-4.2.1-5.fc28 results in binaries
that work. This worries me a little because it looks like a toolchain
interaction.

When I come back online I'll keep trying to fix this.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [modularity] Modularizing the world fast and iteratively

[Carlos O'Donell]

On 09/07/2017 01:50 PM, Alexander Bokovoy wrote:
> The per-symbol API versioning in RPM was proposed five years ago by
> ALT Linux people. It actually works well in their RPM fork.
> 
> An isolated version of that code is available at https://github.com/svpv/rpmss
> 
> You may want to read http://rpm5.org/community/rpm-devel/5356.html and
> associated thread discussion to understand how it all works, but
> overall it is much better suited to cover per-symbol export and import
> between ELF objects. The ABI requirements become explicit and ability to
> automatically detect which applications are using which symbols from a
> library affected by a CVE is a good example where this could be used.

Also of interest...

Bug 1320954 - rpm: Support equivalent of Debian's symbols files for 
finer-grained ELF dependencies
https://bugzilla.redhat.com/show_bug.cgi?id=1320954

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Providing ABI/API assurances for the base runtime in Fedora.

[Carlos O'Donell]

On 09/06/2017 03:50 AM, Petr Pisar wrote:
> On 2017-09-06, Petr Pisar  wrote:
>> Does it mean that when a need for a new feature arises, e.g. adding
>> getrandom(3) into glibc
>>  we will produce
>> a new platform stream distinct from the GA stream?
>>
> I will answer to myself: Yes, we will because modules should be fully
> compatible inside one stream to allow users to downgrade to a previous
> stream release (e.g. in case of a regression).

Correct.

In traditional Fedora the glibc release only adds new symbols during GA.

The GA process is an event which we use to add new symbols, and then
freeze the API/ABI.

Therefore you can move between installed glibc versions in a given Fedora
GA without loosing compatibility.

In a modular world I believe this equates to a stream of the platform module.

I expect that at every glibc upstream release we will probably want to create
a new platform stream with the new glibc symbols. This would also provide a new
platform-glibc to use for linking against to get the new symbols, but we 
wouldn't
have to do that if we didn't want to.

Other modules will then move to the newer platform stream when they are ready,
but once on that newer stream, with a newer platform-glibc, you can't go back.

However, your stream that keeps providing platform-glibc, can actually move
glibc forward to any version it wants since platform-glibc prevents you from
using any of the new functionality. So we could deploy performance improvements
without deploying new features.

So you could stay on the older platform, with an older platform-glibc, but a
newer glibc if needed.
 
> This is not something we support in classical Fedora (because we do not
> retain previous releases in repositories. Will we support it in modular
> Fedora?).

It was always my opinion that *if* Fedora's content delivery network retained
old versions of glibc, then you *could* downgrade to them within the range
of versions associated with your Fedora release. For example you could pick
any number of the glibc releases that came out for F26 on a given F26 install.

With the split I'm suggesting for interface/implementation you could move
your installed glibc to any release you want that is no older than the
platform-glibc package used to build all of your installed packages.

Does that answer your questions?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Providing ABI/API assurances for the base runtime in Fedora.

[Carlos O'Donell]

On 09/04/2017 07:31 AM, Petr Pisar wrote:
> On 2017-09-01, Carlos O'Donell  wrote:
>> I've written up some of the key ideas here:
>> https://fedoraproject.org/wiki/BaseRuntimeInterface
>>
>> Any feedback would be appreciated, including bikeshed on component
>> name prefix for frozen interface pakcages e.g. base-*.

Thank you for your feedback!

> What if there is a bug in the behavior of the frozen base-* packages? Do
> we have to live with the bug for the rest of the life time of the base
> (=platform)? Or do we break this rule and replace the broken package
> with a patched one?

We get this question a lot in glibc.

There is a natural tension to want to break the API/ABI in a stable release
to fix a bug that is known.

In general we live by an "exceptions based policy" which is to say that we
follow the stable API/ABI rules very strictly, but the community can be
convinced to grant an exception.

I think if the platform module releases with a base-glibc package (defines
the interface to build against) that has a bug, say it has the wrong ABI
for a given function, then you need to look at the specifics of the failure
to decide if base-glibc should be fixed and rebuilt.

I added a Q&A question about this. Please tell me if that answers your
question.

> If the second one is the answer, do we know how it will affect packages
> whose build script does run-time checks (i.e. every ./configure script)
> to tune compile-time options?

It depends on how each package implements the split of interface/implementation.

> More strictly speaking, will we be adding new features into the same stream of
> the base module? I guess we won't. Otherwise we have to update the
> frozen base-* packages accordingly to make the new features available at
> build-time of packages that want to use the new features.

The interface base-* or platform-* (whatever we call them) packages would remain
frozen for the lifetime of say the platform modules specific major version.
However, we could rebase the implementation package without specifically 
impacting
the existing components.

In the case of glibc, because we alter the linker path, package configure checks
will only ever see the interface glibc provided by the platform-glibc package.
At present the platform-glibc is a full glibc which can run in the system.

If we were to trim platform-glibc into just some stripped DSOs then we might run
into configure check problems. We would have to make all such configure checks
be cross-compile safe, which is not something I'm going to target at this stage,
but it would be an interesting project to consider.

Does that answer your question?

I added another Q&A item for this.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Providing ABI/API assurances for the base runtime in Fedora.

[Carlos O'Donell]

On 09/01/2017 10:00 AM, Igor Gnatenko wrote:
> On Fri, 2017-09-01 at 09:28 -0500, Carlos O'Donell wrote:
>> Fedora Developers,
> 
>> I am working on a way to provide concrete ABI/API assurances for
>> parts of the base runtime.
> Note that Base Runtime is F26-only thing and in F27 it is called Host
> and Platform[0]..

Thanks! I've updated the proposal based on the new split of host, platform,
and bootstrap.

>> I've written up some of the key ideas here:
>> https://fedoraproject.org/wiki/BaseRuntimeInterface
> 
>> Any feedback would be appreciated, including bikeshed on component
>> name prefix for frozen interface pakcages e.g. base-*.

> Why do we need separate components? Can't we adjust main packages to
> install into separate paths? In theory, it should be just setting
> differeng %_prefix variable for RPM.

I have added a "Questions" section to address this.

There are two reasons to split the packages completely:

* Reduce validation cost for each build e.g. build and validate the interface
  fewer times.

* Reduce storage costs e.g. build the interface only once for each important
  transition point including package rebase.
 
> Another thing to mention is that libabigail currently (as far as I
> know) doesn't handle python code/python extensions (read DNF). Also I
> guess it doesn't handle GObject Introspected code.

Correct, Python and any code that does dynamic introspection (which causes
cross-compilation problems) are currently out of scope for such separation.

libabigail can only look at the binary artifacts (DSOs) which would be generated
from a Python build and characterize those.

In the case of GObject code, I already knew that we could not support such
separation with that kind of code. Effectively what I am suggesting is a small
amount of cross-compilation compatibility with the low-level runtimes in order
that we can upgrade them if required in the Platform module.

Does that answer your questions?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Providing ABI/API assurances for the base runtime in Fedora.

[Carlos O'Donell]

Fedora Developers,

I am working on a way to provide concrete ABI/API assurances for
parts of the base runtime.

I've written up some of the key ideas here:
https://fedoraproject.org/wiki/BaseRuntimeInterface

Any feedback would be appreciated, including bikeshed on component
name prefix for frozen interface pakcages e.g. base-*.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/13/2017 07:01 AM, Carlos O'Donell wrote:
> On 07/12/2017 08:10 PM, Carlos O'Donell wrote:
>> On 07/11/2017 09:48 PM, Carlos O'Donell wrote:
>>> On 07/11/2017 12:37 AM, Carlos O'Donell wrote:
>>>> If we're lucky we can get everything ready for the 12th, but we might
>>>> need another day or two given how long it takes to build gcc on all
>>>> the arches.
>>>
>>> We are getting more luck. We'll see how tomorrow goes.
>>
>> ETA for mass rebuild readiness tomorrow morning EDT after jcajka finishes
>> the final golang rebuild with a fixed glibc (ppc64le failures fixed).
> 
> Status update:
> 
> * Get the gcc trunk patch into F27 gcc. [Done]
> 
> * Rebuild glibc [Done]
> 
> * Fix Go 1.9.0 [In progress... | jcajka]
> 
>   * Build in progress.
> https://koji.fedoraproject.org/koji/taskinfo?taskID=20494275
> 

Status update:

Time is 9:00am EDT.

* gcc - Ready. Confirmed by jakub.

* glibc - Ready. Confirmed by myself.

* golang - Ready. Confirmed by jcajka.

We are ready for mass rebuild.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/12/2017 08:10 PM, Carlos O'Donell wrote:
> On 07/11/2017 09:48 PM, Carlos O'Donell wrote:
>> On 07/11/2017 12:37 AM, Carlos O'Donell wrote:
>>> If we're lucky we can get everything ready for the 12th, but we might
>>> need another day or two given how long it takes to build gcc on all
>>> the arches.
>>
>> We are getting more luck. We'll see how tomorrow goes.
> 
> ETA for mass rebuild readiness tomorrow morning EDT after jcajka finishes
> the final golang rebuild with a fixed glibc (ppc64le failures fixed).

Status update:

* Get the gcc trunk patch into F27 gcc. [Done]

* Rebuild glibc [Done]

* Fix Go 1.9.0 [In progress... | jcajka]

  * Build in progress.
https://koji.fedoraproject.org/koji/taskinfo?taskID=20494275

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/12/2017 08:10 PM, Carlos O'Donell wrote:
> Note:
> - I'm doing a local rebuild of golang 1.9.0 with the new glibc looking for 
> failures
>   before jacjka wakes up tomorrow in the European time zone.
 
This test shows we have one last ppc64le failure for goland 1.9.0
which I've notified jcajka about :-)

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/11/2017 09:48 PM, Carlos O'Donell wrote:
> On 07/11/2017 12:37 AM, Carlos O'Donell wrote:
>> If we're lucky we can get everything ready for the 12th, but we might
>> need another day or two given how long it takes to build gcc on all
>> the arches.
> 
> We are getting more luck. We'll see how tomorrow goes.

ETA for mass rebuild readiness tomorrow morning EDT after jcajka finishes
the final golang rebuild with a fixed glibc (ppc64le failures fixed).

Less lucky and more lucky. The 32-bit ARM bug was solved overnight after
I pinged Nick Clifton. Which is *awesome* otherwise I'd be stuck 
debugging 32-bit ARM code. All that is left is to wait for glibc final
builds to finish and rebuild Go 1.9.0, then we're ready for the mass rebuild.

Status update:

* Get the gcc trunk patch into F27 gcc. [Done]
 
  * Patch committed to trunk [Done -- IBM]

* Fedora backport [Done -- Red Hat]

* Rebuilds of glibc to fix gcc header issues [Done -- fweimer]

* Rebuild gcc and libgcc [Done -- jakub]
  https://koji.fedoraproject.org/koji/taskinfo?taskID=20462687

* Rebuild glibc

  * Patch committed to trunk [Done -- IBM]
 
  * Sync to Fedora [Done -- Red Hat]

  * Rebuild [In progress -- Red Hat]
* Hit a snag with 32-bit ARM.
  * Coordinated with Nick Clifton (bintuils) and Jiong Wang.
  * Fixed here: 
https://www.sourceware.org/ml/libc-alpha/2017-07/msg00518.html
* Scratch build relatively clean.
* Final build in-progress here:
  https://koji.fedoraproject.org/koji/taskinfo?taskID=20482917 

* Fix Go 1.9.0 for s390x
   
  * Fix s390x breakage [Done -- jcajka]
https://bugzilla.redhat.com/show_bug.cgi?id=1460254

  * Rebuild golang with glibc fixes to get ppc64le builds to pass [Waiting -- 
jcajka]
* Once glibc finishes building we rebuild this.

Note:
- I'm doing a local rebuild of golang 1.9.0 with the new glibc looking for 
failures
  before jacjka wakes up tomorrow in the European time zone.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/12/2017 03:34 AM, Dan Horák wrote:
>> * Fix Go 1.8.1 for s390x
> 
> this is https://bugzilla.redhat.com/show_bug.cgi?id=1460254 -
> buggy interaction between golang and new binutils

That's right, we knew what it was, but fixing it is not that
easy since it's generic binutils machinery that should be
working correctly.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/11/2017 12:37 AM, Carlos O'Donell wrote:
> If we're lucky we can get everything ready for the 12th, but we might
> need another day or two given how long it takes to build gcc on all
> the arches.

We are getting more luck. We'll see how tomorrow goes.

Status update:

* Get the gcc trunk patch into F27 gcc.
 
  * Patch committed to trunk [Done -- IBM]

* Fedora backport [Done -- Red Hat]

* Rebuilds of glibc to fix gcc header issues [Done -- fweimer]

* Rebuild gcc and libgcc [In progress -- jakub]
  https://koji.fedoraproject.org/koji/taskinfo?taskID=20462687

* Rebuild glibc
 
  * Patch committed to trunk [In progress -- IBM]
* codonell talking to IBM about fix for static linking.
 
  * Sync to Fedora [Waiting -- Red Hat]
* Waiting for new compiler to retest glibc builds.

  * Rebuild [Waiting -- Red Hat]

* Fix Go 1.8.1 for s390x

  * Recent patches get us to everything but s390x working:
golang 1.8.1: https://koji.fedoraproject.org/koji/taskinfo?taskID=20461040
golang 1.9:   https://koji.fedoraproject.org/koji/taskinfo?taskID=20457700
Note: ppc64le are just testsuite issues.

  * Fix s390x breakage [In progress -- jcajka]

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: super-drafty F28 and F29 schedules

[Carlos O'Donell]

On 07/06/2017 09:15 PM, Matthew Miller wrote:
> https://fedoraproject.org/wiki/Releases/28/Schedule

I encourage Jeff Law and Jakub Jelinek to review these schedules
for compiler related issues.

This is just a perfunctory review from the glibc perspective with
regard to base ABI and API issues in this core runtime.

2018-01-10 MASS REBUILD
- This date is ahead of the glibc release on 2018-02-01
  but after the 2018-01-01 ABI freeze date for glibc.
  Therefore you will be largely assured a stable ABI upon
  which to base Fedora, _but_ there is a vanishingly small
  chance you may need a mass rebuild or a targeted rebuild
  on 2018-02-01 if something has to be reverted.

> https://fedoraproject.org/wiki/Releases/29/Schedule

2018-07-11 MASS REBUILD
- This date is ahead of the glibc release on 2018-08-01
  but after the 2018-07-01 ABI freeze date for glibc.
  Therefore you will be largely assured a stable ABI upon
  which to base Fedora, _but_ there is a vanishingly small
  chance you may need a mass rebuild or a targeted rebuild
  on 2018-08-01 if something has to be reverted.

Otherwise the schedules look sensible from a glibc perspective.
We can drive new security, performance, and usability into the
distribution on these 6 month cycles.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/07/2017 04:20 PM, Carlos O'Donell wrote:
> On 07/07/2017 08:53 AM, Florian Weimer wrote:
>> We currently have an invalid IFUNC resolver in libgcc.a on POWER
>> (rhbz#1467526).  glibc in rawhide recently started linking that into the
>> library and there are significant problems with that (rhbz#1467518).
>>
>> I'll be on PTO next week, and it does not seem likely that this is going
>> to be resolved upstream before that.  If upstream (both glibc GCC are
>> involved) don't come up with a solution, I'm not sure
>>
>> If we downgrade glibc to 2.25 in rawhide, we will have to rebuild at
>> least the following packages right after that because they have a
>> GLIBC_2.26 symbol version dependency:
>>
>>  inn-2.6.1-6.fc27.src.rpm
>>  remctl-3.13-4.fc27.src.rpm
>>  sudo-1.8.20p2-1.fc27.src.rpm
>>  tmux-2.5-1.fc27.src.rpm
>>  unbound-1.6.4-1.fc27.src.rpm
>>  xorg-x11-server-1.19.3-6.fc27.src.rpm
>>
>> Besides the glibc downgrade, I'm not aware of anything else we could do
>> to get ready for the mass rebuild in time.
> 
> IBM has worked through a patch that will fix the libgcc issue:
> https://sourceware.org/ml/libc-alpha/2017-06/msg01430.html
> 
> All we need to do now is:
> * Get the gcc trunk patch into F27 gcc.
> * Rebuild gcc and libgcc.
> * Rebuild glibc.
> 
> And that should be everything we need to do before the mass rebuild
> start on the 12th.
> 
> I'm reaching out to Jakub/Marek on the gcc side to get help with this.

Status update:

* Get the gcc trunk patch into F27 gcc.

  * Patch committed to trunk [Done -- IBM]

  * Rebuild gcc and libgcc [In progress -- Red Hat]

* Blocked on several header issues and changes which make gcc FTBS.
  Working on fixing the issues.
 
  * Fedora backport [Not started]

* Rebuild gcc and libgcc [Not started]

* Rebuild glibc

  * Patch committed to trunk [In progress -- IBM]

* Second round review complete, and final patch in progress.

  * Sync to Fedora [Waiting -- Red Hat]

  * Rebuild [Waiting -- Red Hat]

If we're lucky we can get everything ready for the 12th, but we might
need another day or two given how long it takes to build gcc on all
the arches.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Fedora 27 mass rebuild at risk

[Carlos O'Donell]

On 07/07/2017 08:53 AM, Florian Weimer wrote:
> We currently have an invalid IFUNC resolver in libgcc.a on POWER
> (rhbz#1467526).  glibc in rawhide recently started linking that into the
> library and there are significant problems with that (rhbz#1467518).
> 
> I'll be on PTO next week, and it does not seem likely that this is going
> to be resolved upstream before that.  If upstream (both glibc GCC are
> involved) don't come up with a solution, I'm not sure
> 
> If we downgrade glibc to 2.25 in rawhide, we will have to rebuild at
> least the following packages right after that because they have a
> GLIBC_2.26 symbol version dependency:
> 
>  inn-2.6.1-6.fc27.src.rpm
>  remctl-3.13-4.fc27.src.rpm
>  sudo-1.8.20p2-1.fc27.src.rpm
>  tmux-2.5-1.fc27.src.rpm
>  unbound-1.6.4-1.fc27.src.rpm
>  xorg-x11-server-1.19.3-6.fc27.src.rpm
> 
> Besides the glibc downgrade, I'm not aware of anything else we could do
> to get ready for the mass rebuild in time.

IBM has worked through a patch that will fix the libgcc issue:
https://sourceware.org/ml/libc-alpha/2017-06/msg01430.html

All we need to do now is:
* Get the gcc trunk patch into F27 gcc.
* Rebuild gcc and libgcc.
* Rebuild glibc.

And that should be everything we need to do before the mass rebuild
start on the 12th.

I'm reaching out to Jakub/Marek on the gcc side to get help with this.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: RFH: Annotating ELF binaries

[Carlos O'Donell]

On 01/20/2017 11:55 AM, H.J. Lu wrote:
> We can classify properties into 2 categories: used by run-time loader,
> not used by run-time loader.  We put properties for run-time loader into
> .note.gnu.property section and the rest into GNU attribute section.

Agreed.

Can we use the same noun/adjective for our names?

Is there any reason to use property over attribute?

As a Friday bikeshed I suggest:

.note.gnu.attributes - GNU Attributes optimized for the dynamic loader.
-- New. Follows H.J's proposal. Bit-level, packed, and optimized for the
   dynamic loader processing.

.gnu.attributes - GNU Attributes optimized for offline and static linker
  processing.
-- Existing section. Discussions with Nick ongoing if we can continue
   to use existing infrastructure e.g. Tag_Range to extend this data.

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: RFH: Annotating ELF binaries

[Carlos O'Donell]

On 01/18/2017 12:02 PM, Nick Clifton wrote:
> Hi Carlos,
> 
>> I've added 2 questions to the Toolchain/Watermark wiki but will post them
>> here for posterity:
> 
> Thanks - I'll try answering them here first, and if my answers make sense
> then I will update the wiki.
> 
>> (1) What happened to SHT_GNU_ATTRIBUTES and how does it relate to what 
>> you are proposing?
> 
> Good question, and unfortunately I do not know the answer.  The problem
> is, I have been unable to locate any documentation that describes 
> SHT_GNU_ATTRIBUTES and how it is supposed to be used.
> 
> I think that the two schemes are quite similar, although this new proposal
> is intended to be able to cope with attributes that only apply to part of
> an executable and not necessarily the executable as a whole.  (Also, IMHO,
> my proposal has better documentation...)
 
I'm including Joseph Myers in this discussion.

It's very very well documented by ARM.

The point of SHT_GNU_ATTRIBUTES is to mark a section as containing
GNU-based Attributes for the object in question. Rather than have a generic
SHT_NOTE. The GNU-based attributes can be applied to the file, section, or
symbol (and extended further).

Attributes can be anything you want them to be. Today most of the GNU-attributes
are ABI related and the static linker uses them at link time to issue errors
when objects of mixed incompatible ABIs are attempted to be used together.
That is to say they are Tag_File-based attributes (see binutils source for 
this).

The generic section which holds attributes is SHT_GNU_ATTRIBUTES, but machines
can override this e.g. SHT_ARM_ATTRIBUTES for ARM-specific attributes.

The ELF attributes section design was originally based on the need at the time 
from ARM for ARM EABI Attributes.

See 4.3.6 Build Attributes:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0044f/IHI0044F_aaelf.pdf

.gnu.attributes (section)

* , currently 'A' (0x41)
*  (4 byte unsigned int in ELF endian order of all 
subsequent data)
* "vendor name" ("GNU") for generic + NULL pointer.
*   (all things which are file scope) 

*  (all things which are section scope)  
 
*  (all things which are symbol scope)  
 

The ARM documentation goes into a lot of detail about what attributes it
supports.

Given that we have a GNU-generic version of this... might we not extend things
to a new Tag_Range and keep all the existing machinery we already have in 
binutils
for processing all of these attributes?

Again, the SHT_GNU_ATTRIBUTES section was never designed to be loaded by the
dynamic linker, and it's the reason I recommend you look at the design and think
about how we can avoid another attributes format that is incompatible with the
one we already have in binutils.

Harmonize the designs please :-)

>> (2) What is being done to ensure the attributes are space and time
>> efficient for dynamic link comparison in the dynamic linker? 
>> Speed of checking 10,000 DSOs (scalability) for ABI compatibility is 
>> going to be a very important requirement.
> 
> I believe that H.J's design for the dynamic link notes does take efficiency
> into consideration, but I will leave that for him to comment on further.

It does, mostly be using bits, and AND/OR/EQ handling.

I have yet to see exactly how it works.

I might suggest we version the section like .gnu.attributes so we can
increment the version if we have a flag day changing the layout.

> One thing that I have already done for the static notes is to implement a
> new option for objcopy called "--merge-notes" which eliminates redundancies.
> Theoretically this option could be extended to work with the dynamic notes
> too, helping to make them as space efficient as possible.

OK.

> Another possibility is that the linker could be extended so that when it
> creates a dynamic executable it also inserts a "master" dynamic linker note,
> which contains all of the information that the dynamic linker will need,
> without it having to search through all of the shared libraries used by the
> application.  (This does assume that the shared libraries examined at static
> link time are the same ones that are loaded/used at dynamic link time).
 
We can't assume that unfortunately.
 
>> (b) Loadable notes and space/time efficiency vs. non-loadable notes and
>> static analysis tools.
>>
>> Run-time checking of properties is radically different from offline
>> checking of properties and we absolutely need two different designs to
>> meet these needs. However, if we could weld the two together in a compatible
>> way, that would be great. For example if the dynamic loader could map from
>> a 'run-time property' to a 'link-time property' to increase the verbosity
>> of the error in a failure scenario, then that might be beneficial.
> 
> I think that this might not be easy to do in a way that both imposes a low
> code-increase cost on the dynamic linker and a 
> 

Re: Proposal: Rethink Fedora multilib support (Take Two!)

[Carlos O'Donell]

On 01/09/2017 08:18 PM, Richard W.M. Jones wrote:
> On Mon, Jan 09, 2017 at 09:20:20AM +0100, Jakub Jelinek wrote:
>> On Sat, Jan 07, 2017 at 11:15:28PM +, Richard W.M. Jones wrote:
>>> On Fri, Jan 06, 2017 at 02:47:35AM +0100, Pavel Raiskup wrote:
 On Thursday, January 5, 2017 5:08:16 PM CET Stephen Gallagher wrote:
> Two suggestions were raised as alternatives to the container approach:
>
> * Switch to using the Debian style of multi-arch layout, which instead of
> /usr/lib and /usr/lib64 uses /usr/lib/$ARCH-linux-gnu. Benefits to this 
> would
> include the emergence of a de-facto standard for system layout between 
> the major
> distributions.

 Isn't this just result of good marketing of "multi-arch" distros?  Because
 I fail to see where that approach is superior compared to what we have.
>>>
>>> Partly because there exist more than 2 architectures (think:
>>> RV64G/RV64GC/RV128G, ARMv5/6/7/8, or less esoterically, having various
>>
>> Not all of ARM v5/6/7/8 is ABI incompatible.  The FHS way of using suffixes
>> for */lib is able to deal with more than 2 multilibs, e.g. MIPS has
>> 3 I think.  And ISA flags you meantion (SSE, AVX) should not be separate
>> multilib, those are just optimizations you can do in the same multilib, that
>> can and should be resolved either completely inside of libraries that want
>> to have optimized parts (using IFUNC, target_clones, ...)
> 
> I should note that RV64G vs RV64GC (compressed) is not something that
> could be handled by ifuncs.  It's a deep change that affects all the
> generated code.  I'm hoping that every other RISC-V extension _can_ be
> handled only using ifuncs/target_clones etc.

Could you clarify why IFUNC doesn't work?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: RFH: Annotating ELF binaries

[Carlos O'Donell]

On 01/16/2017 09:37 AM, Nick Clifton wrote:
> Hi H.J.
> 
>> We have 2 different proposals for program properties.  Mine:
>>
>> https://sourceware.org/ml/gnu-gabi/2016-q4/msg00025.html
>>
>> has a much smaller scope.  New features on upcoming Intel platforms,
>> like 5-level paging, need this extension for loader decision at run-time.
>> How should we move forward with program property extensions?
> 
> I would like to combine the two approaches.  Ie use your notes for
> properties that need to be examined at run-time (specifically the
> loader, although I imagine that the application itself might be 
> interested in reading its own notes).  Plus use the note scheme I
> am proposing for static analysis tools.
> 
> I am currently using a gcc plugin to generate the notes, and I think
> that this should be extendable to generate your notes as well.  (Using
> a plugin is advantageous in that it is not tied to the latest gcc release.
> It can be built to run with older gcc's, and it can be updated 
> independently of the gcc release cycle).
> 
> What do you think ?

I've added 2 questions to the Toolchain/Watermark wiki but will post them
here for posterity:

(1) What happened to SHT_GNU_ATTRIBUTES and how does it relate to what 
you are proposing?
 
(2) What is being done to ensure the attributes are space and time
efficient for dynamic link comparison in the dynamic linker? 
Speed of checking 10,000 DSOs (scalability) for ABI compatibility is 
going to be a very important requirement.

Comments:

(a) Ian requests clear separation between language and psABI notes.

Notes are notes. The attribute system should be sufficiently flexible to
handle both, and the "clear sepration" is just a documentation aspect,
and still important, but just about writing down what the notes mean in
clear detail.

(b) Loadable notes and space/time efficiency vs. non-loadable notes and
static analysis tools.

Run-time checking of properties is radically different from offline
checking of properties and we absolutely need two different designs to
meet these needs. However, if we could weld the two together in a compatible
way, that would be great. For example if the dynamic loader could map from
a 'run-time property' to a 'link-time property' to increase the verbosity
of the error in a failure scenario, then that might be beneficial. If we
could translate 'link-time notes' into 'a collection of run-time properties' in
a semi-automatic fashion given strict rules about the notes application,
then that would also be awesome.

(c) The case against SHT_GNU_ATTRIBUTES (Question 2).

Not used. -- Then we should just use them for x86.

IFUNC complication. -- Any new framework must be able to tolerate that
a given interface may have a "range" of ABIs it works with, and those
represent the set of ABIs it can change to. Attributes that don't allow
sets of values are going to be problematic in the face of IFUNCs.

No loadable segment. -- Correct. They were designed for link-time support only.

Most attributes don't apply to dynamic loading. -- Correct. Space inefficient.

Layout not optimal for loading.  -- Correct. Time/Space inefficient.

In summary SHT_GNU_ATTRIBUTES might not work for run-time properties, but
what about link-time properties? Why not resuse this framework?

-- 
Cheers,
Carlos.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: Hacks for multilib unclean C headers

[Carlos O'Donell]

On 06/08/2016 10:21 AM, Jonathan Wakely wrote:
>> If this is so, I will go to FPC with request the ammend the C guidelines
>> with explicit discourage of %{?_isa} on gcc because the main
>> architecture supports the secondary targets and because gcc is not
>> multilib safe.
> 
> I think -devel packages should not "Require: gcc" at all. Not with
> %_isa and not without it.

Agreed.

> To actually compile something using those headers you need the C (or
> C++) build environment installed anyway, i.e. you need 'gcc' (or
> 'gcc-c++') installed so there's no benefit to the package giving it as
> a requirement.

Agreed.

> If you want to install the foobar-devel.i686 package on x86_64 and
> compile 32-bit software with those 32-bit headers you'll need to
> manually install glibc-headers.i686, but that's true even if you just
> want to compile a "Hello, world!" as a 32-bit C program on x86_64.

Right. It's a "specific and special need" which we should clarify in
the "C and C++" application specific guideline.

> So simply don't add a Requires to -devel packages for the build env.
> Don't Require: gcc and don't Require: glibc-headers. If a -devel
> package currently has Requires: glibc-headers%{?_isa} then IMHO that
> Requires should be removed, not replaced with gcc or gcc%{?_isa}
> (unless for some special reason the package really does need
> glibc-headers explicitly, not just in order to provide a working build
> env).

Agreed. Last year I filed bugs against all packages I found that were using
"Require: glibc-headers" and we are about 60% complete the fixes.

https://bugzilla.redhat.com/show_bug.cgi?id=1230505

> I'll update the draft for V2 of the C/C++ Packaging Guidelines to say
> that.

Awesome!

-- 
Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: No Rich boolean deps in Requires/Recommends for f24

[Carlos O'Donell]

On 04/01/2016 02:51 PM, Kevin Fenzi wrote:
> Greetings. 
> 
> Recently FPC oked the use of rich boolean deps. However, we have run
> into an issue where the tools used to push updates are not able to
> correctly handle these new dependencies.
> 
> At today's FESCo meeting we decided to ask maintainers to not use rich
> boolean Requires/Recommends for the time being until tooling can catch
> up and allow us to push updates with them. 

Just to be clear.

Rich deps in Supplements are OK (yum parser ignores those).

And the language pack implementation needs no changes (uses rich deps
in supplements).

-- 
Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/26/2016 04:47 AM, Carlos O'Donell wrote:
> Bug 1238406 - Glibc locale subpackaging
> https://bugzilla.redhat.com/show_bug.cgi?id=1238406
> 
> Changes/Glibc locale subpackaging
> https://fedoraproject.org//wiki/Changes/Glibc_locale_subpackaging

With glibc-2.23.90-3.fc25 and glibc-2.23.1-5.fc24 we have now fixed
the issue in several important ways.

The `glibc-all-langpacks` continues to be the correct way to get all
locales, but now things are a bit smoother.

(1) Virtual provides to make upgrade smoother.

All glibc languages packs now provide `glibc-langpack`, that includes
the glibc-all-langpacks (all language support).

The core runtime depends on `glibc-langpack` and so when upgrading
on systems without dnf, yum, or langpack plugin support, plain rpm
dependency solving via suggests will install `glibc-all-langpack`
for you and provide all the existing locales you previously had.

This makes for a complete rpm solution with no other support
required.

This means that upgrading from a non-langpack supporting F23
to any future langpack supporting distribution will upgrade
you to the newest set of supported locales.

(2) Optimized "all languages" performance.

In addition the `glibc-all-langpack` is optimized to again use
the locale cache (locale-archive) to accelerate lookups for those
users who prefer to have all languages installed. Previously we
had `glibc-all-langpack` as a meta-package of all the other language
packs, but that is no longer the case.

(3) Provide glibc-minimal-langpack

We add a minimal language pack that provides no languages. This
allows you to install glibc-minimal-langpack and then uninstall
glibc-all-langpacks and operate entirely under C, POSIX, and C.UTF-8
locales.

This should resolve all problems that users have been seeing with
the upgrade.

Thank you for your patience. We really messed up here. In the future
we'll make sure we cover all the upgrade cases more carefully.

(4) Remove all languages to minimize installed size

You can still minimize your installed size, but it's slighltly
harder now. First you must install a valid language pack, and then
remove `glibc-all-langpacks` to reduce the size of installed languages.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/29/2016 01:18 PM, Florian Weimer wrote:
> On 02/29/2016 05:18 PM, Adam Williamson wrote:
> 
>> IIRC it would simply terminate claiming the remote host had dropped the
>> connection. There wasn't any useful information even with -. I just
>> kinda guessed it must have something to do with the langpacks,
>> installed glibc-langpack-en, and it started working again immediately.
>> I can't reproduce the problem with 'LC_ALL=C ssh (somewhere)', so I
>> can't immediately dig into it any more.
> 
> Could you try with “LC_ALL=de_DE.UTF-8 ssh …”?  Assuming that
> de_DE.UTF-8 is not installed on the target.
> 
> Other distributions already lack a full complement of locales in default
> installations, and most software works reasonably well in the presence
> of unsupported locale settings.  It's certainly something we need to
> iron out.

Agreed. The similar gnome-terminal failure should be fixed e.g. 
https://bugzilla.redhat.com/show_bug.cgi?id=1312690

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/29/2016 04:22 AM, Vít Ondruch wrote:
> 
> Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
>> On 02/26/2016 05:02 AM, Vít Ondruch wrote:
>>> You also forget to mention what Rawhide users are supposed to do.
>>>
>>> I assume that I should do:
>>>
>>> # dnf install langpacks-cs
>>>
>>> to have only czech language available and English should be available by
>>> default, right?
>> No languages are available by default, we did this because otherwise
>> you keep carrying forward locales that you can't remove. So for F24
>> you get nothing by default, and dnf langpacks plugin does everything
>> for you in the background.
>>
>> If you're on rawhide, and you don't use langpacks, you can install
>> the appropriate langpack by hand with `dnf install glibc-langpack-cs`.
> 
> This is very badly communicated I must say. I really don't understand,
> why the suggested steps are not:
> 
> 1) dnf install langpack-[your-language]
> 2) dnf update glibc\*
> 
> This should be the workflow and not "you use" or "you don't use". This
> is not something which will be glibc specific. The
> langpack-[your-language] is precondition to make this work, not to
> suggest installing glibc-langapack-[your-language] or even
> glibc-all-langpacks.
> 
> I am bit annoyed by this, because doing just "dnf update" will break the
> system in a way that gnome-terminal won't start anymore, DNF is totally
> busted etc. so simple instructions would be really appreciated.

I agree, and I apologize. We're working on an rpm dependency based resolution
process which will fix this shortly.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/29/2016 10:46 AM, Richard W.M. Jones wrote:
> On Mon, Feb 29, 2016 at 01:42:57PM +, Richard W.M. Jones wrote:
>> On Sat, Feb 27, 2016 at 08:57:27PM +, Igor Gnatenko wrote:
>>> And also I was not able to start gnome-terminal and some other apps. I had
>>> to install glibc-langpack-en.
>>
>> libvirtd also fails to start up:
>>
>>   https://bugzilla.redhat.com/show_bug.cgi?id=1312688
>>
>> I'm investigating it now.
> 
> Is it OK to ignore the return value of setlocale()?

Yes and no.

It's a quality issue.

Without the locale your localizations like dates may be in
formats the user doesn't expect.

It's up to you to decide what quality you consider "critical."

We are going to fix this shortly in F24+ though so that more
locales are installed during the upgrade.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/27/2016 08:33 PM, Adam Williamson wrote:
> On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
>> The glibc in Fedora rawhide and F24 is now split by
>> language packs. We have over 180 supported languages
>> in glibc, and those have been split into langpacks
>> for transparent install and support via dnf. This
>> greatly reduces the size of a glibc install from 130MB
>> down to a couple of megs. It drops support for the hacky
>> %_inst_langs feature, and relies entirely on langpack
>> support (it was one or the other).
> 
> Also in unpredictable fallout: on upgraded systems, ssh stops working
> until you install a langpack. At least it did for me.

That should never happen. What failed?

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/27/2016 03:44 PM, Adam Williamson wrote:
> On Fri, 2016-02-26 at 04:47 -0500, Carlos O'Donell wrote:
>> The glibc in Fedora rawhide and F24 is now split by
>> language packs. We have over 180 supported languages
>> in glibc, and those have been split into langpacks
>> for transparent install and support via dnf. This
>> greatly reduces the size of a glibc install from 130MB
>> down to a couple of megs. It drops support for the hacky
>> %_inst_langs feature, and relies entirely on langpack
>> support (it was one or the other).
>>
>> Thanks to Mike Fabian for the great work!
> 
> This did completely break anaconda (and thus also broke live image and
> Cloud image creation).
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1312607
> 

The glibc team is working to put out an ASAP update that
would use a slightly tweaked set of rpm dependencies to
ensure glibc-all-lanpacks is pulled in by default in more cases.

We appreciate your patience.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/29/2016 06:32 AM, Petr Pisar wrote:
> On 2016-02-27, Ruben Kerkhof  wrote:
>> I'm now seeing this in mock / koji builds:
>>
>> perl: warning: Setting locale failed.
>> perl: warning: Please check that your locale settings:
>> LANGUAGE = (unset),
>> LC_ALL = (unset),
>> LANG = "en_US.UTF-8"
>> are supported and installed on your system.
>> perl: warning: Falling back to the standard locale ("C").
>>
> Mock (or rpmbuild?) sets LANG to en_US.UTF-8 instead of C in order to
> have UTF-8 locale when building packages because it was decided that
> UTF-8 is the default (build) environment for Fedora.
> 
> With removing en_US.UTF-8 from the glibc and thus minimal build root,
> all koji builds are misconfigured now because setlocale(3) on "en_US.UTF-8"
> fails without the locale definition in the file system.
> 
> I assume changing the default build-time locale to C.UTF-8 could help.

The glibc team guarantees you will have C.UTF-8 always.

It should be the default.

This is indeed related to the langpack split up and we are going
to fix this shortly.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/26/2016 10:38 AM, Michael Catanzaro wrote:
> On Fri, 2016-02-26 at 10:06 -0500, Carlos O'Donell wrote:
>> Correct, 'glibc-all-langpacks' will solve the problem immediately.
> 
> OK, I've added this to comps, to take care of new installs. If anyone
> implements support for language pack installation in both gnome-
> initial-setup and gnome-control-center, we can remove this.
> 
>> We're working right now on the solution Stephen suggested which is
>> a weak ref/suggests which looks like it would let rpm handle the
>> upgrade properly.
> 
> And I trust you guys will take care of upgrades. Thanks!

We've also had some questions about the performance and disk space
of the split up langpacks. So I'm going to work on a blog post to
help clarify this for everyone.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/26/2016 10:39 AM, Zbigniew Jędrzejewski-Szmek wrote:
> On Fri, Feb 26, 2016 at 08:56:27AM -0500, Stephen Gallagher wrote:
>> On 02/26/2016 08:39 AM, Miroslav Suchý wrote:
>>> Dne 26.2.2016 v 11:20 Carlos O'Donell napsal(a):
>>>> No languages are available by default, we did this because otherwise
>>>> you keep carrying forward locales that you can't remove.
>>>
>>> For this reasons we have weak dependencies. I guess that:
>>>   Recommends: glibc-all-langpacks
>>> can do the work (install or by default, can be safely removed). Or
>>>   Suggests: glibc-all-langpacks
>>> so it is not installed by default, but users can get the information that 
>>> it can enhance the usability of glibc.
>>>
>>
>>
>> Yeah, I think the best approach would be to have all the langpacks offer a
>> virtual Provides: glibc-langpack and have the main package Requires:
>> glibc-langpack and Suggests: glibc-all-langpacks. The net result would be 
>> that
>> unless a specific langpack was chosen, you'd end up with all of them to 
>> satisfy
>> the requirement. (This would also unbreak the upgrades without needing a 
>> patch
>> to dnf system-upgrade)
> 
> That would be a much better solution. dnf-system-upgrade so far didn't
> have any special handling for specific packages (except for a check
> that the kernel is in the upgrade transaction) and there no mechanism
> like this is implemented.

We are investigating switching to this. It shouldn't be a problem, and
with a quick rebuild of glibc in F24 I think we'll be ready. I just need
to run through the testing quickly on Monday.

All of our langpacks are auto-generated based on glibc supported locales
so there is no grunt work in changing, just testing again that it works
smoothly.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/26/2016 09:44 AM, Michael Catanzaro wrote:
> On Fri, 2016-02-26 at 12:25 +, Richard Hughes wrote:
>> What happens if the user isn't using DNF? For the workstation we have
>> to support users using just the graphical tools, and we can't rely on
>> command line tools for this kind of thing. I'm happy to add support
>> for installing lang packs into gnome-software but need to know how
>> the
>> langpack plugin works so we can add proper support into the AppStream
>> metadata file. The idea being if the user changes the per-session
>> language we install the required langpacks automatically.
>>
>> Can someone explain how the dnf langpack plugin gets the data on what
>> application subpackages to install? It would be easy if we could have
>> a virtual package that we could install for each locale as then we
>> can
>> just put the metainfo file there for easy support in gnome-software.
> 
> We also need locale selection in gnome-control-center and gnome-
> initial-setup to work. Any chance you'll work on that too, Richard?
> 
> Otherwise, we should be able to resolve this by adding glibc-all-
> langpacks to the Workstation group in comps (correct Carlos?). It
> defeats the point of this change, but releasing with broken locale
> selection is not gonna happen.

Correct, 'glibc-all-langpacks' will solve the problem immediately.

We're working right now on the solution Stephen suggested which is
a weak ref/suggests which looks like it would let rpm handle the
upgrade properly.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/26/2016 07:33 AM, Parag Nemade wrote:
> Hi,
> 
> On Fri, Feb 26, 2016 at 3:52 PM, Carlos O'Donell  wrote:
>> On 02/26/2016 04:47 AM, Carlos O'Donell wrote:
>>> The glibc in Fedora rawhide and F24 is now split by
>>> language packs. We have over 180 supported languages
>>> in glibc, and those have been split into langpacks
>>> for transparent install and support via dnf. This
>>> greatly reduces the size of a glibc install from 130MB
>>> down to a couple of megs. It drops support for the hacky
>>> %_inst_langs feature, and relies entirely on langpack
>>> support (it was one or the other).
>>
>> Unfortunately we missed a piece of the puzzle here.
>>
>> Bug 1303034 - rpm macro expansion works incorrectly
>>   when looping over a long list using lua
>> https://bugzilla.redhat.com/show_bug.cgi?id=1303034
>>
>> We need this fixed in F24 and pushed into the buildroots
>> to unblock building glibc.
> 
> I thought this is already fixed in F24. At least I remember I tested
> langpacks glibc.spec from mfabian's some old private git repository on
> F24 machine. The rpm build I tested was rpm-4.13.0-0.rc1.23.fc24. But
> now I see that f24 glibc is failing while comparing the SUPPORTED file
> in %prep.

We didn't get that far in last nights build. I'll go look immediately.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/26/2016 04:47 AM, Carlos O'Donell wrote:
> The glibc in Fedora rawhide and F24 is now split by
> language packs. We have over 180 supported languages
> in glibc, and those have been split into langpacks
> for transparent install and support via dnf. This
> greatly reduces the size of a glibc install from 130MB
> down to a couple of megs. It drops support for the hacky
> %_inst_langs feature, and relies entirely on langpack
> support (it was one or the other).

Unfortunately we missed a piece of the puzzle here.

Bug 1303034 - rpm macro expansion works incorrectly 
  when looping over a long list using lua
https://bugzilla.redhat.com/show_bug.cgi?id=1303034

We need this fixed in F24 and pushed into the buildroots
to unblock building glibc.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

On 02/26/2016 05:02 AM, Vít Ondruch wrote:
> You also forget to mention what Rawhide users are supposed to do.
> 
> I assume that I should do:
> 
> # dnf install langpacks-cs
> 
> to have only czech language available and English should be available by
> default, right?

No languages are available by default, we did this because otherwise
you keep carrying forward locales that you can't remove. So for F24
you get nothing by default, and dnf langpacks plugin does everything
for you in the background.

If you're on rawhide, and you don't use langpacks, you can install
the appropriate langpack by hand with `dnf install glibc-langpack-cs`.

Cheers,
Carlos
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

glibc in Fedora rawhide now split by langpacks.

[Carlos O'Donell]

The glibc in Fedora rawhide and F24 is now split by
language packs. We have over 180 supported languages
in glibc, and those have been split into langpacks
for transparent install and support via dnf. This
greatly reduces the size of a glibc install from 130MB
down to a couple of megs. It drops support for the hacky
%_inst_langs feature, and relies entirely on langpack
support (it was one or the other).

Thanks to Mike Fabian for the great work!

Bug 1238406 - Glibc locale subpackaging
https://bugzilla.redhat.com/show_bug.cgi?id=1238406

Changes/Glibc locale subpackaging
https://fedoraproject.org//wiki/Changes/Glibc_locale_subpackaging

Notes:
- If you don't use langpacks you won't get any locales
  except C, POSIX and C.UTF-8.
- If you don't want to use langpacks just install the
  glibc langpack you want manually.
  e.g. dnf install glibc-langpack-en
- Anaconda should take care of everything for you via
  langpacks.
- We missed the fact that for the F23->F24 transition
  `dnf system-upgrade` should install glibc-all-langpacks
  (meta package) to give F23 users a smooth transition
  with all the locales they had before, but once in F24
  the langpacks will be used.
  Filed:
  Bug 1312103 - “dnf system-upgrade” needs to install 
  “glibc-all-langpacks” when upgrading from f23 to f24
  https://bugzilla.redhat.com/show_bug.cgi?id=1312103

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: [GCC6] fatal error: You must enable NEON instructions (e.g. -mfloat-abi=softfp -mfpu=neon) to use these intrinsics.

[Carlos O'Donell]

On 02/18/2016 08:37 PM, Kevin Kofler wrote:
> Peter Robinson wrote:
>> It's an issue we see occasionally where the package thinks it knows
>> better than the explicit CFLAGs being set, I'll get it sorted out.
> 
> But why are those intrinsics now requiring NEON at all? Those are GCC byte 
> swap intrinsics documented as being architecture-independent and should be 
> implemented in software (through shifts or rotates) when there is no native 
> byte swap instruction.
> 
> And is Fedora actually requiring NEON as the baseline on ARM? I thought it 
> was not.

We require an ARMv7A or newer and a VFP with enough registers to support
the hard ABI e.g. -march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16

The message in the header error is only an example.

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

dnf --refresh upgrade vs. CVE-2015-7547 updates.

[Carlos O'Donell]

On 2016-02-16 16:41:27 the glibc rawhide build to fix CVE-2015-7547[1] 
completed:
https://koji.fedoraproject.org/koji/buildinfo?buildID=736361

On 2016-02-17 the rawhide repodiff email showed the build:
~~~
glibc-2.22.90-36.fc24 
- 
* Tue Feb 16 2016 CArlos O'Donell https://www.sourceware.org/ml/libc-alpha/2016-02/msg00416.html
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: [LLVM] CommandLine Error: Option 'track-memory' registered more than once!

[Carlos O'Donell]

On 02/17/2016 03:24 AM, Igor Gnatenko wrote:
> Hi guys, I see this error when building POCL or Beignet which looks
> like a bug in clang or llvm, can you please investigate?
> 
> : CommandLine Error: Option ': CommandLine Error: Option
> 'track-memory' registered more than once!
> track-memory' registered more than once!
> LLVM ERROR: inconsistency in registered CommandLine options
> LLVM ERROR: inconsistency in registered CommandLine options
> : CommandLine Error: Option 'track-memory' registered more than once!
> LLVM ERROR: inconsistency in registered CommandLine options
> 
> Reference: 
> https://kojipkgs.fedoraproject.org//work/tasks/3359/13013359/build.log

I talked to ajax about this?

The llvm DSOs are being loaded and unloaded, but not all of them, so you get
them being registered more than once.

I was roped in because it was considered a possible duplicate constructor
call in the loaded DSO, but it's not.

The solution is to fix clam/llvm or stop loading/unloading your own framework.

Does that make any sense?

Adam, Did I remember this right?

Cheers,
Carlos.
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

PSA: Statically linking against the C/C++ runtime is not supported.

[Carlos O'Donell]

Follow the Fedora Guidelines on static linking against
a library. If you really need to use static linking,
you should know that we can not support statically linking
against the C/C++ runtime.

This includes the use of the gcc, clang, or ld option:

   -static
   -static-.* e.g. -static-libstdc++

to link against any part of the C/C++ runtime[1].

This also includes the use of the Go build options:

   -ldflags '-extldflags=-static -linkmode=external'

to create a go application that would otherwise have been
dynamically linked but are now statically linked against
the C/C++ runtime[2].

Note that in RHEL the glibc-static package is in the optional
repository and therefore also unsupported by virtue of this
caveat.

You may be able to statically link against other runtimes 
e.g. "-Wl,-static -lfoo -Wl,-Bdynamic" to link just "foo" 
statically, but please review the documentation for the 
specific runtime to see if it supports this use (and don't
forget the Fedora packaging guidelines).

In all honesty we want to support some limited APIs in static
linkage, but until we document this better we can't easily
commit to any given API. The set of unsupported APIs includes
but is not limited to:

   Any glibc functionality which might call dlopen.
   - All NSS functions e.g. getent, getpwuid_r, etc.
   - All encoding conversions (gconv/iconv).
   - All IDN functionality (libidn).
   - All dl* functions (libdl).
   - Thread cancellation.

The primary reason for our inability to support static
linking is the same reason we have problems implementing
dlmopen (from Solaris) correctly. Namely that the static and
shared parts (those loaded by dlopen) are two distinct
namespaces and we have yet to design infrastructure to
share the process-global state that both namespaces need
in order to coordinate properly. The v2 C/C++ packaging
guidelines has more help and guidance for developers who
really need help making static linking with the C/C++
runtime safe.

In Fedora and CentOS there is no optional repository
distinction, and in these distributions we are working
on a long-term solution to this problem. A short term
solution is likely a documentation of the APIs which can
be used in a static link.

There are a few cases of core distribution binaries like
sln and ldconfig which must be statically linked. In those
cases we use our best judgement and experience to prevent
those binaries from deviating into unsupported APIs as
described above.

Lastly, keep in mind that this problem is not unique to
the C/C++ runtimes. The same problem is faced by all
libraries that may be statically linked and dlopen'd.
Such libraries face the the fact that they have been
loaded twice into the same process (or N times with
dlmopen load isolation) and must either operate
correctly or provide their own APIs for process-global
state coordination.

I am updating the C/C++ packaging guidelines to in [3]
with a new section on static linking. Once the section has
had enough review it will be put forward for FPC to vote and
update the official C/C++ packaging guidlines in [4].

Cheers,
Carlos.

[1] For the pedants in the crowd this does not include the
implementation dependent archives which may or may not be
statically linked into each object e.g. libc_nonshared.a,
libpthread_nonshared.a etc, these are
implementation-dependent details.

[2] The complexity of this caveat is because if the application
would otherwise have been statically linked it means the
go compiler determined there were no external API uses
and defaulted to linking in the required Go runtime
statically (which is a supported use case, and is not a
static linkage against the C/C++ runtime).

[3] https://fedoraproject.org/wiki/C_and_C%2B%2B_v2#Static_Linking

[4] https://fedoraproject.org/wiki/Packaging:C_and_C%2B%2B
--
devel mailing list
devel@lists.fedoraproject.org
http://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Fedora Rawhide glibc-2.22.90-6.fc24 shipping C.UTF-8 locale.

[Carlos O'Donell]

Thanks to some great work by Mike FABIAN the Rawhide glibc is 
now shipping with a C.UTF-8 locale for use as a default UTF-8
locale. The locale can be referenced as "C.UTF-8" or "C.utf8".
Any feedback on the locale is greatly appreciated.

The locale is added to the pre-processed memory mapped locale
archive (/usr/lib/locale/locale-archive) which means there is
little to no performance impact for having this locale. However,
in order to make sure that the locale is robustly available in
all scenarios, including one in which a locale archive update
fails, the locale is added as /usr/lib/locale/C.utf8. Yes, this
means that the locale consumes 2x1.8MB on disk for this initial
implementation.

I consider this a real step forward to providing all the upper
stacks with a default UTF-8 locale that we can guarantee will
always be available.

Comments welcome.

Cheers,
Carlos.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Proposal to reduce anti-bundling requirements

[Carlos O'Donell]

On 09/14/2015 08:29 AM, Florian Weimer wrote:
> On 09/11/2015 04:34 PM, Carlos O'Donell wrote:
> 
>>> How do you propose to resolve symbol conflicts if both the system
>>> library and the bundled library are loaded into the same process?  The
>>> current ELF linking scheme lacks good support for bundling libraries
>>> whose exported symbols have not been mangled in some way.
> 
>> If you need different versions of the same shared libraries in the same
>> process then you must use dlmopen to load the conflicting set into a
>> distinct load namespace.
>>
>> At present dlmopen is not functional for this purpose in upstream
>> glibc master. It is something that I'm working on fixing [1].
> 
> I looked at the Solaris documentation, and I'm not sure if it's the
> right use-case.  This seems to provide complete isolation, and would
> break things like SQLite (at least older versions without file-private
> locks) which need process-global state.

Correct, with dlmopen you have complete isolation, but I don't understand
the SQLite issue, so if you could expand on the requirement I'm happy
to listen.

> I think the real issue here is the ELF model with backwards/forwards
> linking and symbol interposition.  Ideally, we should load each DSO
> exactly once, resolve object symbols only against explicit DT_NEEDED
> dependencies (not indirect dependencies), and make most symbols
> non-interposable by default.  At least this is my gut feeling.  This is
> a very difficult problem, especially at distribution scale.

I'm not sure how difficult this would be because I'm not sure how many
symbols rely upon indirect dependencies. I think it would be a worthwhile
cleanup to turn on something like you suggest, and attempt to bootstrap
the OS using Fedora Bootstrap [1].

> We currently do not perform proper symbol namespace management in Fedora
> (as we discussed before).  Perhaps we should try to track DSO symbol
> namespaces first, and use that data to guide further evolution of
> dynamic linking.
 
Agreed. We do indeed need some infrastructure in tools to extract all
symbols out of the entire distribution and review them.

Cheers,
Carlos.

[1] http://fedora-bootstrap.osop.rhcloud.com/ (currently down?)
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Fedora Ring 0 definition

[Carlos O'Donell]

On 09/14/2015 05:10 PM, Brendan Conoboy wrote:
> You are right that we do need to think about overall goals to be
> achieved, then the policies that achieve those goals.  For my part I
> am interested in distinguishing the OS from the applications that run
> on top of it.  This might be the difference between ring 0 and ring
> 1.  It's too early to tell, though.  During today's base wg we talked
> a little bit further about possible goals that a ring 0 might fulfill
> (http://meetbot.fedoraproject.org/fedora-meeting-2/2015-09-14/fedora_base_design_working_group.2015-09-14-14.15.log.html).
> A few hypothetical possibilities as examples:
> 
> 1. Make a "Base" (or ring0) compose who has its own alpha/beta/ga
> cycle that precedes the RC deadlines for the current editions and
> spins, providing a stable set of NVRs to base upon.
> 
> 2. New boundaries for primary/secondary arch blocker status, rules
> for excludearch, threshold for inclusion in primary koji, etc.
> 
> 3. Decouple the ring 0 release cycle and support terms from the
> editions.  Maybe base comes out every 4 months. Or 9 months.  Maybe
> it's supported for 24 months.  Things like that- it's small.
> Editions and spins can pick the base release to build on.

+1

These all sound like great ideas.

Cheers,
Carlos.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Proposal to reduce anti-bundling requirements

[Carlos O'Donell]

On 09/10/2015 10:42 AM, Florian Weimer wrote:
> On 09/10/2015 03:53 PM, Stephen Gallagher wrote:
> 
>> I would like to propose that the no-bundled-libraries policy be
>> amended  as follows: "Any package that has an existing mechanism to
>> link against a shared system library and functions correctly when
>> doing so must link against that library and not bundle it internally.
>> Any package whose upstream releases cannot link against a shared
>> system library (or are incompatible with the version in Fedora) may
>> bundle that library (without requiring a special exemption) but MUST
>> add Provides: bundled() =  in the spec file for each
>> known bundled library.(This will allow us to track down the bundling
>> when we need to). Package maintainers should continue attempt to
>> engage upstream to support linking against shared system libraries
>> wherever possible, due to the advantages it provides the package
>> maintainer."
> 
> Is  the name of the SRPM which provides the system version of
> the library?
> 
> How do you propose to resolve symbol conflicts if both the system
> library and the bundled library are loaded into the same process?  The
> current ELF linking scheme lacks good support for bundling libraries
> whose exported symbols have not been mangled in some way.
> 

If you need different versions of the same shared libraries in the same
process then you must use dlmopen to load the conflicting set into a
distinct load namespace.

At present dlmopen is not functional for this purpose in upstream
glibc master. It is something that I'm working on fixing [1].

Failing that we need to implement Solaris' grouping model (RTLD_GROUP,
-B group, RTLD_PARENT) to further support isolation at the dynamic
loader level.

All of this would be useful for software collections, and better
handling of multiple installed versions of libraries, all while
having the potential to avoid containers and bundling.

Cheers,
Carlos.

[1] https://sourceware.org/ml/libc-alpha/2015-07/msg00474.html
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Proposal to reduce anti-bundling requirements

[Carlos O'Donell]

On 09/10/2015 03:42 PM, Adam Miller wrote:
>> It doesn't matter how rare they are, it'll only take a single bundled
>> library handled incorrectly to completely screw a running OS. I don't
>> think this is something that can just be swept under the carpet, it
>> needs to be addressed as a core part of the proposal and currently is
>> not.
>>
> 
> I was talking to someone at Red Hat Summit from the GCC/glibc team and
> I was under the impression they (upstream) are working on something
> that handles the symbol tables to allow parallel installed versions of
> libs. No idea how far this is along but it's something that people are
> working towards.

You spoke to me, and I am responding here to clarify exactly what the
Fedora glibc team trying to do.

The work being done is to enable an alternate linkage model for Ring-0,
where the implementation and interface of ELF-based shared libraries is
split in two. Linking is carried out against a curated Ring-0 SDK
and ensures if we upgrade say, glibc, and have accidentally added a
new symbol, that no new package can use said symbol until the Ring-0 SDK
package is updated to also have that symbol. Updating the Ring-0 SDK
requires careful review and would likely be frozen after GA. The Ring-0
SDK contains headers, stub libraries, and more.

While this idea of an SDK could be extended to all ELF-based shared
libraries in the distribution, it gets much harder outside of Ring-0
and is more work with less benefit. Keep in mind the goal here is to
provide automation and tooling to measure and control the most important
core part of the OS, that layer upon which the other layers rely.

The idea itself could allow you to install a brand new glibc on your
system without impacting any builds you do with rpmbuild, mockbuild,
etc. You are isolated from the implementation by the new linkage
model (using curated sysroots). Thus with the brand new glibc, and
because glibs offers backwards compatibility, you could run newer ISV
applications that require newer glibc, without needing a VM. This
ignores several problems like bug-for-bug compatibility, subtle changes
that alter undefined behaviour or unspecified behaviour, and eventually
result in the failure of an application to run against a new glibc.

However, with the new linkage model it is possible to use it to upgrade,
not not run multiple versions, of shared libraries.

If you need different versions of shared libraries then you need to use
software collections for those libraries (load isolation via the dynamic
loader), containers (load isolation via namespaces), VMs (load isolation
at the kernel level).

If you need different versions of the same shared libraries in the same
process then you must use dlmopen for symbol namespace isolation. Note
that at present dlmopen is not functional for this purpose in upstream
glibc master.

Cheers,
Carlos.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Symbol `SSL_ImplementedCiphers' has different size in shared object, consider re-linking

[Carlos O'Donell]

On 08/28/2015 03:23 PM, Josh Stone wrote:
> I update from nss-3.19.3-1.0.fc22.x86_64 to nss-3.20.0-1.0.fc22.x86_64
> this morning, and now I get this stderr output:
> 
> $ /usr/bin/stap -V >/dev/null
> /usr/bin/stap: Symbol `SSL_ImplementedCiphers' has different size in
> shared object, consider re-linking
> 
> The message comes from ld.so; that symbol comes from libssl3.so.
> 
> Should I be worried about this?  Do we need a rebuild of all
> nss-dependent packages to clear this message?

Well, it's an ABI break. If you care about ABI issues then the change
causing the breakage needs reverting and the broken packages need to
be rebuilt.

c.
 

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

FYI: Building glibc rpm using --with bootstrap.

[Carlos O'Donell]

I just finished extending the `--with bootstrap` support for
rawhide glibc (rawhide glibc is FTBS right now, but that's another
reason). The goal is to provide RCM and others in Fedora with the
ability to experiment with bootstraps using the standard set of
rpm tooling.

When bootstrap is active we now:

- No documentation. (--without docs)
- No more dependencies on texinfo in this case.

- Skip Valgrind smoke tests. (--without valgrind)
- Rely on normal regression tests only.

- Drop SELinux support
- No dependency on SELinux.
- Remember this is a bootstrap, so we allow some
  flexibility to be less secure. In this case it
  only impacts nscd.

- Drop NSS Crypt support
- Bootstrapping will never get us FIPS compliance
  so we drop NSS Crypt support and fall back to the
  normal crypt routines.

- Disable the use of -Werror (--without werror)
- This prevents build failure due to newer compiler
  or other dependent headers.

and lastly...

- Disable building microbenchmarks. (--without benchtests)
- The microbenchmark binary subpackage that can be
  used to do comparitive runs is disabled.

I encourage others to continue to support `--with bootstrap`,
particularly in the low-level tools. At present our stage1
Fedora boostrap scripts use custom recipies, but if we continue
to extend `--with bootstrap` it may be possible to support
this entirely within the normal set of rpm tooling.

Comments welcome.

Cheers,
Carlos.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Removing glibc librtkaio support in Fedora Rawhide.

[Carlos O'Donell]

On 08/21/2015 03:23 PM, Carlos O'Donell wrote:
> * Immediately remove rtkaio from Fedora Rawhide, deprecating the library,
>   and providing a system change notification about the library removal for
>   F24.

Both the glibc 2.23 and librtkaio removal system change notifications have
now been drafted and will be ready for Fedora 24.

https://fedoraproject.org/wiki/Changes/GLIBC223

https://fedoraproject.org/wiki/Changes/GLIBC223_librtkaio_removal

Cheers,
Carlos.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Removing glibc librtkaio support in Fedora Rawhide.

[Carlos O'Donell]

On July 2003 the rtkaio add-on was added to Fedora in glibc 2.3.2-64.
The rtkaio add-on provided a POSIX realtime API interface that used 
linux kernel Asynchronous IO support (KAIO) to provide high performance 
AIO for a small subset of files (those using O_DIRECT, and not all 
file types). Typically the use case was databases or high-speed
transactional systems that needed fast AIO. The libraries were installed
under /lib64/rtkaio/ e.g. /lib64/rtkaio/librt.so.1 (symlink to
/lib64/rtkaio/librtkaio-X.Y.so with SONAME librt.so.1) and could be used
by preloading (LD_PRELOAD), dynamic linker lookup path changes
(LD_LIBRARY_PATH) or by directly opening the shared library (dlopen).
This accelerated access to file was used by customers like Sybase during
the development of their database Sybase ASE (now owned by SAP).

It has been 12 years since rtkaio was released, and while it has seen
some uptake, the only known usage example is Sybase ASE. Sybase now
exclusively uses the Linux Kernel Asynchronous I/O Library (libaio) for
over 10 years ago and no longer use rtkaio. The libaio project provides
a unique API that is tailored to doing very fast AIO. An analysis of
Fedora shows no packages using rtkaio. Lastly the rtkaio add-on was
never contributed upstream, likely because it never provided full POSIX
conformance and worked only with a small subset of the required POSIX
realtime features, those supported by KAIO.

It is the conclusion of the Fedora glibc team that the maintenance burden
of rtkaio is no longer warranted. The glibc team suggest rtkaio be deprecated
and removed. Application developers should use libaio if they want high
performance KAIO, or use librt if they want portable and flexible AIO.

What are the consequences of removing rtkaio?

* Application developers using rtkaio will see a performance decrease if 
  they were previously using KAIO on O_DIRECT opened files, but should
  otherwise see no semantic changes in their applications.

* Application developers using LD_PRELOAD will see a warning that the
  preloaded library is missing, but the application will load the normal
  librt and continue to operate correctly.

* Application developers using LD_LIBRARY_PATH will see no warning, and
  the application will load the normal librt and continue to operate correctly.

* Application developers using dlopen will see a failure from dlopen since
  the library is missing. This is mitigated by shipping a symlink from the
  rtkaio library to librt in the official Fedora release.

The rtkaio library and the librt library are ABI and API compatible, and
therefore interchangeable. As long as we provide one of them we will meet
our application compatibility requirements. We will continue to provide the
POSIX realtime library (librt) forever.

The following plan of action is suggested:

* Immediately remove rtkaio from Fedora Rawhide, deprecating the library,
  and providing a system change notification about the library removal for
  F24.

  https://bugzilla.redhat.com/show_bug.cgi?id=1227855
  
  No compatibility symlinks are provided for Rawhide. We want to use Rawhide
  to detect if any applications are actually using rtkaio. Before the F24
  branch we will add a symlink.

Cheers,
Carlos.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: F23 System Wide Change: Glibc locale subpackaging

[Carlos O'Donell]

On 06/22/2015 10:59 AM, Alexander Larsson wrote:
> On mån, 2015-06-22 at 06:16 -0400, Jan Kurik wrote:
>> = Proposed System Wide Change: Glibc locale subpackaging  =
>> https://fedoraproject.org/wiki/Changes/Glibc_locale_subpackaging
>>
>> Change owner(s):
>> * Mike Fabian 
>> * Siddhesh Poyarekar 
>> * Carlos O’Donell 
>>
>> This change should make it possible to install or uninstall locales 
>> individually. 
>>
>> == Detailed Description ==
>> Currently the file /usr/lib/locale/locale-archive contains all 
>> locales and is thus huge (103MB).
>> For small systems (and containers) it would be useful to be able to 
>> install only a small number of locales.
>> Recently we made it possible to install a small number of locales by 
>> supplying the rpm-macro “_install_langs”, for example
>>
>>rpm -i -D _install_langs="en:de_DE" glibc-common.rpm
>>
>> will install all English locales and all German locales which start 
>> with “de_DE”,
>>
>>rpm -i -D _install_langs="en_US.utf8" glibc-common.rpm
>>
>> will install only the en_US.utf8 locale,
>>
>>rpm -i -D _install_langs="POSIX" glibc-common.rpm
>>
>> will install nothing (but the POSIX/C is still available because it 
>> is builtin into glibc).
>>
>> But this approach works only during an Anaconda based install when 
>> Anaconda supplies the _install_langs rpm-macro.
>> When glibc is updated later, the _install_langs macro will not be 
>> supplied on the command line during the update and the default value 
>> “all” of “_install_langs” from /usr/lib/rpm/macros will be used and 
>> all locales come back during an update.
>> Therefore, this solution is far from perfect.
>> It should be made possible to install and uninstall locales 
>> individually, for example by having a separate package for the 
>> locales for each language. Installing such a package would add these 
>> locales to locale-archive, uninstalling it would remove them.
> 
> Do they really have to modify locale-archive? Can't each package
> install separate archive files (say, based on the locale name).
> Packaging optional extra files is a lot easier for me in my work with
> an xdg-app runtime based on fedora.
 
For now we are modifying locale-archive, but we are aware the this causes
problems with container overlays, and are looking for a solution where
this works on a per-file basis with each new subpackage langauge contributing
a new loadable optimized binary locale file.

The stepping stone will likely be:

_install_langs (to limit languages)
-> subpackages that modify locale-archive 
-> subpackages that install their own files

Please remember the core runtimes are key part of the OS, and we make
these transitions slowly and gather metrics about how effective each
step was and how many bugs we got, and if the intermediate changes
broke anything.

In the end we may just skip the intermediate step depending on testing,
but I won't guarantee that.

Cheers,
Carlos.
 

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct