Re: [ABRT] Intent to put abrt-hook-ccpp to rest
If you no longer plan to add the feature for generating backtraces without writing core files to disk, then abrt-hook-ccpp is pretty much useless - IMHO. -- Původní e-mail -- Od: Ernestas Kulik Komu: devel@lists.fedoraproject.org Datum: 4. 9. 2019 10:06:05 Předmět: [ABRT] Intent to put abrt-hook-ccpp to rest "Hello, fellow humans, Since nowadays pretty much everyone has systemd-coredump in their /proc/sys/kernel/core_pattern[1][2], I would like to remove the custom ABRT hook that does largely the same thing. If anyone has made a conscious decision to keep using it, please let us know. [1] - https://fedoraproject.org/wiki/Changes/coredumpctl [2] - https://abrt.github.io/abrt/systemd/core_pattern/2017/03/07/fedora-26- change/ -- Ernestas Kulik Associate Software Engineer - Base Operating Systems (Core Services/ABRT) Red Hat Czech, s.r.o. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of -conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists. fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: vanishing abrt logs
> I think abrt should figure out if the crashing binary is part of Would it be an option to create your own configuration for libreport? It is much simpler and robust to deliver a config tailored for your packages than trying to generalize the task and define a global policy Example: cat >/etc/libreport/events.d/my_system_service.conf < my_system_service.log EOF > Could we have a per-service switch that says Unfortunately, I am not sure we can say that a coredump file never contains any security sensitive data. For example, it contains a copy of /proc/[pid]/environ and if not cleared and an admin has a strange idea to export a crazy stuff, we are in trouble. Jakub -- Původní e-mail -- Od: Zbigniew Jędrzejewski-Szmek Komu: Development discussions related to Fedora Datum: 9. 4. 2019 9:04:48 Předmět: Re: vanishing abrt logs "On Mon, Apr 08, 2019 at 02:21:47PM +0200, jfi...@fedoraproject.org wrote: > I agree, but the file /proc/meminfo is not present, right? > And yes, the abrt thing just reads the data from journal and > > stores them on filesystem to be able to upload them to Bugzilla. > > > > > > Regarding the missing logs, the journal log lines should > > be extracted by this thing: > > https://github.com/abrt/abrt/blob/master/src/plugins/ccpp_event.conf#L25 > (https://github.com/abrt/abrt/blob/master/src/plugins/ccpp_event.conf#L25) > > which is a little bit complex, so no wonder if it is broken Thanks for the link. That code boils down to "journalctl -q -b --since=-3m --system -n 99 _COMM="$base_executable" and for systemd services this is very underwhelming, because it doesn't use the information that the process is part of service, and all logs from that service are relevant, e.g. when it launches a second executable or switches uid. I think abrt should figure out if the crashing binary is part of a system service, and switch to a log collection mode that is more like 'journalctl -u' in this case. Also, don't limit the messages to just 3 minutes or 99 messages. > (I am just not sure why the abrt test suite is not failing though). > > Regarding the core files, I was under the impression that > the core files can be attached only if the reporter wants to attach it. > IOW the core files were never attached automatically (due to security > issues). Could we have a per-service switch that says "this service never has private information, make the coredump file collection opt-out"? For example, we get a number of reports for systemd-logind, and by design it never has any private user data or keys. The user interface could be simplified. Similarly for hwrngd, and probably many others. > If you need some information that is relevant only to your packages, > we can work together to create a new abrt configuration which will > gather that information for your packages > (for example, dnf ships its own abrt configuration). Can you point me to it? rpm -ql $(rpm -qa|grep dnf)|grep abrt doesn't yield anything. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists. fedoraproject.org " ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: vanishing abrt logs
I agree, but the file /proc/meminfo is not present, right? And yes, the abrt thing just reads the data from journal and stores them on filesystem to be able to upload them to Bugzilla. Regarding the missing logs, the journal log lines should be extracted by this thing: https://github.com/abrt/abrt/blob/master/src/plugins/ccpp_event.conf#L25 (https://github.com/abrt/abrt/blob/master/src/plugins/ccpp_event.conf#L25) which is a little bit complex, so no wonder if it is broken (I am just not sure why the abrt test suite is not failing though). Regarding the core files, I was under the impression that the core files can be attached only if the reporter wants to attach it. IOW the core files were never attached automatically (due to security issues). If you need some information that is relevant only to your packages, we can work together to create a new abrt configuration which will gather that information for your packages (for example, dnf ships its own abrt configuration). Regards, Jakub -- Původní e-mail -- Od: Zbigniew Jędrzejewski-Szmek Komu: Development discussions related to Fedora Datum: 8. 4. 2019 13:49:41 Předmět: Re: vanishing abrt logs "On Mon, Apr 08, 2019 at 01:27:10PM +0200, jfi...@fedoraproject.org wrote: > Hi Zbyszek, > > > > If you want more files attached to ABRT Bugzilla reports, please add them to > systemd-coredump first :) > > https://github.com/systemd/systemd/blob/master/src/coredump/coredump.c#L 1074 > (https://github.com/systemd/systemd/blob/master/src/coredump/coredump.c#L 1074) In a sense, they already *are* attached to systemd-coredump. coredumpctl is basically just a journal query, and all the information I asked for is also present in the journal. abrt already extract *some* logs from the journal, so it's a question of adjusting the filters. The other thing which is missing from abrt reports is the corefile. In the past we used to have that attached. It would be great to have it again. (For example in case of memory structure errors, the backtrace itself is not particularly useful, but a lot could be diagnosed by examining the structures in memory. But I can't look at all reports quickly, and when I ask reporters if they have the core file, after a month or two the answer is usually "not anymore".). I *am* trying to solve the abrt reports that are files for my packages, but it's getting harder to do. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists. fedoraproject.org " ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: vanishing abrt logs
Hi Zbyszek, If you want more files attached to ABRT Bugzilla reports, please add them to systemd-coredump first :) https://github.com/systemd/systemd/blob/master/src/coredump/coredump.c#L1074 (https://github.com/systemd/systemd/blob/master/src/coredump/coredump.c#L1074) so ABRT folks can simply list them in their white-list: https://github.com/abrt/abrt/blob/master/src/plugins/abrt-dump-journal-core. c#L35 (https://github.com/abrt/abrt/blob/master/src/plugins/abrt-dump-journal-core.c#L35) Regards, Jakub PS: I forwarded your email to ABRT developers -- Původní e-mail -- Od: Zbigniew Jędrzejewski-Szmek Komu: Development discussions related to Fedora Datum: 7. 4. 2019 11:14:37 Předmět: vanishing abrt logs "Hi, I was looking at some abrt reports in the bugzilla [1,2,3], and I seems that the reports contain much less information than in the past. In particular, the "var_log_messages" attachment is always just a few useless lines that a) seems to be cut down just to messages *from* the service, and has no messages *about* the service b) the messages are only from a short time period before the crash When a service dies, systemd logs quite a bit of useful information [4] about the causes. We also have services like pcp, which log information about resource starvation [5]. This extra information would surely be helpful to diagnose abrt reports. Can we please make the reports contain useful logs again (at least full 'journalctl -u' for the service and all logs timed from around the crash) ? More information about the machine, for example /proc/meminfo would be great too. Zbyszek [1] https://bugzilla.redhat.com/show_bug.cgi?id=1660466 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1691889 [3] https://bugzilla.redhat.com/show_bug.cgi?id=1697032n [4] $ sudo systemd-run -p WatchdogSec=5 sleep 10 Running as run-re57f9e7da8234592bf74818522ad9644.service... $ journalctl --no-hostname -o short-precise -u run-re57f9e7da8234592bf 74818522ad9644.service -- Logs begin at Tue 2019-02-19 02:28:12 CET, end at Sun 2019-04-07 10:55:09 CEST. -- Apr 07 10:47:02.891163 systemd[1]: Started /usr/bin/sleep 100. Apr 07 10:47:08.071563 systemd[1]: run-re57f9e7da8234592bf74818522ad9644. service: Watchdog timeout (limit 5s)! Apr 07 10:47:08.072333 systemd[1]: run-re57f9e7da8234592bf74818522ad9644. service: Killing process 7704 (sleep) with signal SIGABRT. Apr 07 10:47:08.403477 systemd[1]: run-re57f9e7da8234592bf74818522ad9644. service: Main process exited, code=dumped, status=6/ABRT Apr 07 10:47:08.403608 systemd[1]: run-re57f9e7da8234592bf74818522ad9644. service: Failed with result 'watchdog'. Apr 07 10:47:08.403946 systemd[1]: run-re57f9e7da8234592bf74818522ad9644. service: Consumed 4ms CPU time. [5] pcp-pmie[3000]: Severe demand for real memory 11.2pgsout/s@krowka ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists. fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: vanishing abrt logs
Hi Kevin, I was under the impression that the problem with many emails from Bugzilla has already been fixed: https://bugzilla.redhat.com/show_bug.cgi?id=1660157 (https://bugzilla.redhat.com/show_bug.cgi?id=1660157) https://github.com/abrt/libreport/commit/569bf0e3fed698e93b8e098bf6a0bb2f773 aed6a (https://github.com/abrt/libreport/commit/569bf0e3fed698e93b8e098bf6a0bb2f773aed6a) Regards, Jakub -- Původní e-mail -- Od: Kevin Fenzi Komu: devel@lists.fedoraproject.org Datum: 7. 4. 2019 20:19:40 Předmět: Re: vanishing abrt logs "On 4/7/19 1:59 AM, Zbigniew Jędrzejewski-Szmek wrote: > Hi, > > I was looking at some abrt reports in the bugzilla [1,2,3], and I seems that > the reports contain much less information than in the past. In particular, > the "var_log_messages" attachment is always just a few useless lines that > a) seems to be cut down just to messages *from* the service, and has no > messages *about* the service > b) the messages are only from a short time period before the crash > > When a service dies, systemd logs quite a bit of useful information [4] > about the causes. We also have services like pcp, which log information about > resource starvation [5]. > > This extra information would surely be helpful to diagnose abrt reports. > Can we please make the reports contain useful logs again (at least > full 'journalctl -u' for the service and all logs timed from around the crash) ? > > More information about the machine, for example /proc/meminfo would be > great too. I think: https://github.com/abrt/abrt/ is the correct place to file bugs/RFE's. I don't know if any of the abrt developers read the list or not. I really wish someone could fix libreport to only send 1 or 2 emails on new abrt bugzilla tickets again. It used to do this, but somehow broke with bugzilla5 upgrade, so now when someone files an abrt bug you get like 25 emails for it. I thought I filed a bug on this, but I can't seem to find it right now, will look more and refile if I can't find it. Also, someone noted the other day that abrt/libreport doesn't work with the new signin options on bugzilla5. You login with the redhat or fedora idps and it will ask you still for a username/password to file the bug. :( kevin ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists. fedoraproject.org " ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: F26 System Wide Change: Golang 1.8
Brilliant! I was thinking about creating a proof of concept a posting it as a change myself. However, I'm really busy with other projects, so I might miss F26. Brad's suggestion made me happy, that's why I close the issue. Though, I would not name the flag 'abrt' but 'tracebackcrash' or something like that. Its name should tell users something about its effect. I'm not sure how we package Go projects and how we build them, but my idea is to include "--tags=tracebackcrash" in the build flags when running rpmbuild. That means that we need to work with the rpm team and ask them to update the default macros. Will this change affect also gcc-go? I'm afraid that we need to come up with something similar for gcc-go too, right? Regards, Jakub -- Původní zpráva -- Od: Jakub Cajka Komu: Development discussions related to Fedora Datum: 5. 1. 2017 16:38:03 Předmět: Re: F26 System Wide Change: Golang 1.8 " - Original Message - > From: jfi...@fedoraproject.org > To: "Development discussions related to Fedora" > Sent: Wednesday, December 14, 2016 7:18:32 AM > Subject: Re: F26 System Wide Change: Golang 1.8 > > I agree with Zbyzsek on this. > > What about to carry a tiny down-stream patch until this issue is fixed: > https://github.com/golang/go/issues/18304 > > > Jakub > > Suggestion by Brad Fitzpatrick in the upstream issue seems as reasonable implementation to me. Also it make possible opt-out/opt-in, if there will be need for it. I will work on implementing/testing it along with the re-base(including update to packaging macros and Go guidelines draft) and I will update the change proposal to reflect it ASAP. Sorry for longer notice I have been offline though the end of the year, JC > > > -- Původní zpráva -- > Od: Zbigniew Jędrzejewski-Szmek > Komu: Development discussions related to Fedora > > Datum: 13. 12. 2016 19:35:01 > Předmět: Re: F26 System Wide Change: Golang 1.8 > > > On Tue, Dec 13, 2016 at 01:06:29PM -0500, Jakub Cajka wrote: > > > can we enable coredumping for Go programs by default - i.e. set > > > GOTRACEBACK= > > > crash? > > > > > > Currently, Go terminate a process that panic and prints out an error > > > message on stderr. > > > > > > This approach does not provide much room for automatic Go panic > > > detection. > > > > It should be possible without any significant side effects apart from > > generating cores and traces. But to enable this, I believe, it would need > > alteration to the default system env. > > Would it be possible? What is the package providing the default env vars? > > systemd has DefaultEnvironment= in /etc/systemd/system.conf, but it is > supposed to be used to create local overrides, and doesn't work well > for this case (it's %config(noreplace) among other problems). In general > setting global env vars does not work. > > Instead, it would be nicer to modify the go runtime to default to a coredump > if GOTRACEBACK= is not set. This would cover more cases and would not pollute > the environment for users who are not using go. > > Zbyszek > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Interpreting FAF reports
Isn't this related to no free disk space? I've seen some strange mmap related crashes when a process mmaped a sparse file and disk ran out of free space. Anyway, please do let me know what additional information would help you to resolve the issue. Please bare in mind that FAF report must be anonymous and automatically submitted. Regards, Jakub -- Původní zpráva -- Od: Florian Weimer Komu: Development discussions related to Fedora Datum: 29. 12. 2016 10:29:37 Předmět: Interpreting FAF reports "Any idea what this is about? https://retrace.fedoraproject.org/faf/reports/1154372/ To me that looks like a combination of several factors. First of all, the backtrace generation likely used incorrect debuginfo data because the backtrace is impossible. Stack corruption is unlikely to yield a relatively consistent backtrace—iconv and gconv match up, only the nscd and sunrpc functions in the middle do not make sense. I got lucky and build ID 25ea1fd961cb2f5a38172614365bd4c1aacb01a6 refers to the current version of /usr/lib64/gconv/ISO8859-1.so, so I could do the disassembly manually. The crash is in the gconv function, at address 0xb50: b44: 49 39 d5 cmp %rdx,%r13 b47: 72 2a jb b73 b49: 48 89 d3 mov %rdx,%rbx b4c: 48 83 c0 01 add $0x1,%rax b50: 0f b6 50 ff movzbl -0x1(%rax),%edx b54: 48 39 c5 cmp %rax,%rbp b57: 89 53 fc mov %edx,-0x4(%rbx) b5a: 75 e4 jne b40 b5c: 41 bb 04 00 00 00 mov $0x4,%r11d b62: e9 1a ff ff ff jmpq a81 b67: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1) Experimentation with GDB shows that this is in the part which converts from ISO-8859-1, and it is the load from the input buffer. A crash at this point is impossible because we have a bounds check in the gconv implementation framework before this load. However, iconv (the command) maps the input file, and something is truncating that file, causing the SIGBUS error. This is just how mmap works in POSIX, unfortunately. Is there a way to discover who is submitting these crash reports and what they are trying to do? I wonder if we should remove the mmap from the iconv command, so that we would not crash in this case. Florian ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org " ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Interpreting FAF reports
Hi Sérgio, It depends on what is your goal. Do you want to fix the crash? You can log in to FAF and check if there is a contact email or a comment. If there is not additional information, you can just wait until some opens a Bugzilla bug report. Do you want to get rid of the notification? You need to do that in Fedora Notifications app. Or you can create a Bugzilla bug from the FAF report and close the bug as INSUFFCIENTDATA. FAF will synchorize the state of the FAF report with the Bugzilla bug, so the report will appear CLOSED. However, this way you will never receive a full Bugzilla bug report. I've just started putting together a wiki page that should help maintainers to deal with ABRT. We should have done this several years ago, sorry about that: https://fedoraproject.org/wiki/Automatic_Bug_Reporting_Tool Regards, Jakub -- Původní zpráva -- Od: SérgioBasto Komu: devel@lists.fedoraproject.org Datum: 29. 12. 2016 15:53:51 Předmět: Re: Interpreting FAF reports "BTW I received ABRT report for package mlt has reached 100 occurrences Packages: mlt Function: QObject::disconnect(QObject const*, char const*, QObject const*, char const*) First occurrence: 2016-12-17 Type: core Count:100 URL: http://retrace.fedoraproject.org/faf/reports/bthash/43f48b90184f02 e38f071ebff19966cdb25376cc/ what I can do ? I don't find anything related with mlt package ! On Qui, 2016-12-29 at 10:28 +0100, Florian Weimer wrote: > Any idea what this is about? > > https://retrace.fedoraproject.org/faf/reports/1154372/ > > To me that looks like a combination of several factors. First of > all, > the backtrace generation likely used incorrect debuginfo data > because > the backtrace is impossible. Stack corruption is unlikely to yield > a > relatively consistent backtrace—iconv and gconv match up, only the > nscd > and sunrpc functions in the middle do not make sense. > > I got lucky and build ID 25ea1fd961cb2f5a38172614365bd4c1aacb01a6 > refers > to the current version of /usr/lib64/gconv/ISO8859-1.so, so I could > do > the disassembly manually. > > The crash is in the gconv function, at address 0xb50: > > b44: 49 39 d5cmp%rdx,%r13 > b47: 72 2a jb b73 > b49: 48 89 d3mov%rdx,%rbx > b4c: 48 83 c0 01 add$0x1,%rax > b50: 0f b6 50 ff movzbl -0x1(%rax),%edx > b54: 48 39 c5cmp%rax,%rbp > b57: 89 53 fcmov%edx,-0x4(%rbx) > b5a: 75 e4 jneb40 > b5c: 41 bb 04 00 00 00 mov$0x4,%r11d > b62: e9 1a ff ff ff jmpq a81 > b67: 66 0f 1f 84 00 00 00nopw 0x0(%rax,%rax,1) > > Experimentation with GDB shows that this is in the part which > converts > from ISO-8859-1, and it is the load from the input buffer. A crash > at > this point is impossible because we have a bounds check in the gconv > implementation framework before this load. > > However, iconv (the command) maps the input file, and something is > truncating that file, causing the SIGBUS error. This is just how > mmap > works in POSIX, unfortunately. > > Is there a way to discover who is submitting these crash reports and > what they are trying to do? I wonder if we should remove the mmap > from > the iconv command, so that we would not crash in this case. > > Florian > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org -- Sérgio M. B. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Creating cores in f25
-- Původní zpráva -- Od: Lennart Poettering Komu: Development discussions related to Fedora Datum: 19. 12. 2016 13:51:09 Předmět: Re: Creating cores in f25 "On Mon, 19.12.16 00:07, Tom Hughes (t...@compton.nu) wrote: > On 18/12/16 23:08, Matthew Miller wrote: > > On Sun, Dec 18, 2016 at 01:00:37PM -0600, Michael Catanzaro wrote: > > > > How do I get f25 to create cores, these days? > > > Some more details here: > > > https://fedoraproject.org/wiki/Changes/coredumpctl > > > > I have ulimit -c returning 0 in shells on my system — have I done some > > configuration I don't remember? That's the default, isn't it? Should it > > stay that way with this change? > > The ulimit on core dump size is (mostly) ignored if sysctl has been used to > tell the kernel to send core dumps to a pipe. > > The only exception is that a limit of 1 byte (which is impossible to set in > most shells) effectively acts as a limit of 0 and disables core dumps but > that's only because the kernel uses it internally to ignore core dumps from > the process running the pipe to receive a core dump. Note that "systemd-coredump" will honour the resource limit anyway, even if the kernel ignores it as soon as a core_pattern is installed. I am pretty sure we should honour resource limits if they are set, as they are pretty OK and well-documented way how to enable and disable coredump collection, and well, putting limits on them. Note that abrtd (to my knowledge) ignores the resource limit, and will collect coredumps in anyway if if disabled with it. " ABRT does not ignore the resource limit. If the limit is not 0, ABRT creates a core file according the value of 'sysctl kernel.core_pattern' before it was changed to ABRT pattern. This basically means that ABRT creates core file in prorcess' CWD if the limit (ulimit -c) is not 0. This is the behavior we are used to. However, it is true that ABRT creates a core dump file in /var/spool/abrt/ regardless of the limit. It is possible to tell kernel to skip core dumping at all by setting "RLIMIT _CORE" to 1. Regards, Jakub ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: Creating cores in f25
Hi Steve, Please have a look at this email: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/ message/HQ4JFTYLPT5GRW6AD4M2MWGMRAPE7ITN/ systemd developers has decided to change the default RLIMIT_CORE (ulimit -c) from "0" to unlimited, therefore ABRT must stop creating the core dump files in CWD. Otherwise, you will get core dumps spread all over your file system - that is because of crashes of daemon that used to not crash with core dump file (their RLIMIT_CORE were 0 by default; starting with systemd-229 the default RLIMIT_CORE is unlimited). Regards, Jakub -- Původní zpráva -- Od: Steve Dickson Komu: Development discussions related to Fedora Datum: 18. 12. 2016 17:35:43 Předmět: Creating cores in f25 "Hello, How do I get f25 to create cores, these days? I'm getting the segfault [55108.290610] rpc.gssd[13264]: segfault at 0 ip 55dc90af9dde sp 7f9 fb73cb7c0 error 4 in rpc.gssd[55dc90af3000+14000] but no core so those address are meaningless. Everything in the kernel is set: f25# sysctl -a | grep kernel.core kernel.core_pattern = |/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t %P %I kernel.core_pipe_limit = 4 kernel.core_uses_pid = 1 The abrtd daemon is up and running f25# ps ax | grep abrtd 713 ? Ssl 0:00 /usr/sbin/abrtd -d -s What am I missing? tia, steved. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: F26 System Wide Change: Golang 1.8
I agree with Zbyzsek on this. What about to carry a tiny down-stream patch until this issue is fixed: https://github.com/golang/go/issues/18304 (https://github.com/golang/go/issues/18304) Jakub -- Původní zpráva -- Od: Zbigniew Jędrzejewski-Szmek Komu: Development discussions related to Fedora Datum: 13. 12. 2016 19:35:01 Předmět: Re: F26 System Wide Change: Golang 1.8 "On Tue, Dec 13, 2016 at 01:06:29PM -0500, Jakub Cajka wrote: > > can we enable coredumping for Go programs by default - i.e. set GOTRACEBACK= > > crash? > > > > Currently, Go terminate a process that panic and prints out an error > > message on stderr. > > > > This approach does not provide much room for automatic Go panic detection. > > It should be possible without any significant side effects apart from generating cores and traces. But to enable this, I believe, it would need alteration to the default system env. Would it be possible? What is the package providing the default env vars? systemd has DefaultEnvironment= in /etc/systemd/system.conf, but it is supposed to be used to create local overrides, and doesn't work well for this case (it's %config(noreplace) among other problems). In general setting global env vars does not work. Instead, it would be nicer to modify the go runtime to default to a coredump if GOTRACEBACK= is not set. This would cover more cases and would not pollute the environment for users who are not using go. Zbyszek ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: F26 System Wide Change: Golang 1.8
Oh drat, I was hoping for a build time configuration option: https://github.com/golang/go/issues/18304 Anyway, the env variable could be exported in the /etc/profile file which is owned by the setup package. However, I don't think it is a good idea to place it there. I would rather export it in a file from the /etc/profile.d/ directory. The profile.d file should be delivered by a golang package that is required to run go programs. Regards, Jakub -- Původní zpráva -- Od: Jakub Cajka Komu: jfi...@fedoraproject.org Datum: 13. 12. 2016 19:07:41 Předmět: Re: F26 System Wide Change: Golang 1.8 " - Original Message - > From: jfi...@fedoraproject.org > To: jca...@redhat.com > Cc: devel-annou...@lists.fedoraproject.org, "Development discussions related to Fedora" > > Sent: Tuesday, December 13, 2016 6:17:23 AM > Subject: Re: F26 System Wide Change: Golang 1.8 > > > Jakub, > > > > > can we enable coredumping for Go programs by default - i.e. set GOTRACEBACK= > crash? > > > > > Currently, Go terminate a process that panic and prints out an error > message on stderr. > > This approach does not provide much room for automatic Go panic detection. > > It should be possible without any significant side effects apart from generating cores and traces. But to enable this, I believe, it would need alteration to the default system env. Would it be possible? What is the package providing the default env vars? JC > > > > > > > > > Regards, > > Jakub > > ABRT > > > > > > -- Původní zpráva -- > Od: Jan Kurik > Komu: Development discussions related to Fedora org>, devel-annou...@lists.fedoraproject.org > Datum: 12. 12. 2016 12:32:15 > Předmět: F26 System Wide Change: Golang 1.8 > > "= Proposed System Wide Change: Golang 1.8 = > https://fedoraproject.org/wiki/Changes/golang1.8 > > Change owner(s): > * Jakub Čajka > > Rebase of Golang package to upcoming version 1.8 in Fedora 26, > including rebuild of all dependent packages. > > > == Detailed Description == > Rebase of Golang package to upcoming version 1.8 in Fedora 26. Golang > 1.8 is schedule to be released in Feb. Due to current nature of Go > packages, rebuild of dependent package will be required to pick up the > changes. > > > == Scope == > * Proposal owners: > Rebase golang package in f26, help with resolving possible issues > found during package rebuilds. > > * Other developers: > fix possible issues > > * Release engineering: > As there is scheduled mass-rebuild, nothing should be required. > > * List of deliverables: N/A > > * Policies and guidelines: N/A > > * Trademark approval: N/A > -- > Jan Kuřík > Platform & Fedora Program Manager > Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic > ___ > devel-announce mailing list -- devel-annou...@lists.fedoraproject.org > To unsubscribe send an email to devel-announce-leave@lists.fedoraproject. org > " ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: F26 System Wide Change: Golang 1.8
Jakub, can we enable coredumping for Go programs by default - i.e. set GOTRACEBACK= crash? Currently, Go terminate a process that panic and prints out an error message on stderr. This approach does not provide much room for automatic Go panic detection. Regards, Jakub ABRT -- Původní zpráva -- Od: Jan Kurik Komu: Development discussions related to Fedora , devel-annou...@lists.fedoraproject.org Datum: 12. 12. 2016 12:32:15 Předmět: F26 System Wide Change: Golang 1.8 "= Proposed System Wide Change: Golang 1.8 = https://fedoraproject.org/wiki/Changes/golang1.8 Change owner(s): * Jakub Čajka Rebase of Golang package to upcoming version 1.8 in Fedora 26, including rebuild of all dependent packages. == Detailed Description == Rebase of Golang package to upcoming version 1.8 in Fedora 26. Golang 1.8 is schedule to be released in Feb. Due to current nature of Go packages, rebuild of dependent package will be required to pick up the changes. == Scope == * Proposal owners: Rebase golang package in f26, help with resolving possible issues found during package rebuilds. * Other developers: fix possible issues * Release engineering: As there is scheduled mass-rebuild, nothing should be required. * List of deliverables: N/A * Policies and guidelines: N/A * Trademark approval: N/A -- Jan Kuřík Platform & Fedora Program Manager Red Hat Czech s.r.o., Purkynova 99/71, 612 45 Brno, Czech Republic ___ devel-announce mailing list -- devel-annou...@lists.fedoraproject.org To unsubscribe send an email to devel-announce-le...@lists.fedoraproject.org "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: F26 System Wide Change: Enable coredumpctl by default
-- Původní zpráva -- Od: Lennart Poettering Komu: Development discussions related to Fedora Datum: 6. 12. 2016 11:11:48 Předmět: Re: F26 System Wide Change: Enable coredumpctl by default "On Tue, 06.12.16 10:16, Miroslav Suchý (msu...@redhat.com) wrote: > Dne 6.12.2016 v 09:52 Gerd Hoffmann napsal(a): > > Hmm, isn't this as easy as abrt being able to find and analyze coredumps > > written by coredumpctl (in addition to the coredumps written by the abrt > > dumper)? > > Yes, it is quite easy. But ABRT cannot do this query every second/minute/ hour. So systemd should have mechanism which > notify ABRT that there is new dump (by push event). abrt can watch the journal relatively easily. In essence, abrt should store away a "cursor" (which is a short string that identifies a location in the journal), that indicates the location it last read from. And then when running continously: 1. Open the journal sd_journal_open() 2. restrict the output that you only see coredumps with sd_journal_add_match() specifying a match of: MESSAGE_ID=fc2e22bc6ee647b6b 90729ab34a250b1 2. If abrt previously stored a cursor, use it with sd_journal_seek_cursor() 3. now iterate through the journal: use sd_journal_enumerate_data()/sd_ journal_restart_data() to get any data you want for one specific journal entry, and then sd_journal_next() to skip to the next. 4. For each entry you want the raw naked coredump for: use the data included in the COREUMP field if it is there. If not, there's COREDUMP_FILENAME= which contains the coredump filename. 5. If you reached the end of the journal, use sd_journal_wait() to wait for more entries as they arrive. The man in particular of sd_journal_next() and sd_journal_wait() have longer examples. There's currently no concept of "service activation by coredump". That's mostly because it's quite hard to do in a race-free fashion. It's not sufficient to simply make systemd-coredump call out to abrt, as that ignores that coredumps can happen during early boot and late shutdown, and there hence should be an asynchronous element: processes that happen early on or in the last boot cycle should be processed during the normal runtime. " Thank you. The service watching journald for coredumps saved by systemd- coredump already exists: http://abrt.readthedocs.io/en/latest/examples.html#getting-core-files-from- systemd-coredumctl Jakub " "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: F26 System Wide Change: Enable coredumpctl by default
If you remove ABRT and set kernel.core_pattern to 'core', then if you don't configure systemd to use RLIMIT_CORE=0, every crashing process will create a new core dump file in its CWD: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/ message/HQ4JFTYLPT5GRW6AD4M2MWGMRAPE7ITN/ Jakub -- Původní zpráva -- Od: DJ Delorie Komu: Development discussions related to Fedora Datum: 5. 12. 2016 23:08:56 Předmět: Re: F26 System Wide Change: Enable coredumpctl by default " Zbigniew Jędrzejewski-Szmek writes: > You still can restore such behaviour pretty easily. Just set the > kernel.core_pattern sysctl. Yup, that's what I do. Just adding my two cents on Fedora trying to help me "be a developer" without doing what I need as a developer. But I typically uninstall abrt too. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org " ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: F26 System Wide Change: Enable coredumpctl by default
-- Původní zpráva -- Od: DJ Delorie Komu: Development discussions related to Fedora Datum: 5. 12. 2016 20:37:43 Předmět: Re: F26 System Wide Change: Enable coredumpctl by default " Jan Kurik writes: > Note that coredumpctl is intended as a developer tool, As a developer, I remove abrt and anything else that redirects cores away from my development area. It's really hard to debug a core dump if you can't find the core file." ""-- Původní zpráva --""Od: DJ Delorie "" Komu: Development discussions related to Fedora ""Datum: 5. 12. 2016 20:37:43""Předmět: Re: F26 System Wide Change: Enable coredumpctl by default"" "" ""Jan Kurik writes:""> Note that coredumpctl is intended as a developer tool,"" ""As a developer, I remove abrt and anything else that redirects cores""away from my development area. It's really hard to debug a core dump if""you can' t find the core file."" ""Just sayin'""___""devel mailing list -- devel@lists.fedoraproject.org""To unsubscribe send an email to devel-le...@lists.fedoraproject.org" Just sayin' " " Then you have no problem with this change, because if you remove ABRT, systemd-coredump takes over core dumping :) Regards, Jakub "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Re: F26 System Wide Change: Enable coredumpctl by default
" -- Původní zpráva -- Od: Zbigniew Jędrzejewski-Szmek Komu: Development discussions related to Fedora Datum: 5. 12. 2016 21:57:12 Předmět: Re: F26 System Wide Change: Enable coredumpctl by default "On Mon, Dec 05, 2016 at 02:36:13PM -0500, DJ Delorie wrote: > > Jan Kurik writes: > > Note that coredumpctl is intended as a developer tool, > > As a developer, I remove abrt and anything else that redirects cores > away from my development area. It's really hard to debug a core dump if > you can't find the core file. You still can restore such behaviour pretty easily. Just set the kernel.core_pattern sysctl. " Please don't set kernel.core_pattern to "core" or any path unless you set the default RLIMIT_CORE to 0. Otherwise you will get core dump files littered all around your file system. The default value of RLIMIT_CORE, that was 0 for decades, has been changed to 'unlimited' in systemd-229. " systemd-coredump+coredumpctl give you pretty easy access to core files, they are just dumped into /var/lib/systemd/coredump/. The advantage is that a) things are logged and can be easily looked up and queried, b) you get a lot of metadata like open files, cgroups, /proc/mountinfo, umask, capability masks, etc., c) a stacktrace is automatically generated and logged locally, d) old coredumps are automatically removed, e) you cannot fill your disk with coredumps. " You are describing the ABRT coredumping solution, aren't you? :) No, seriously, I have to admit that coredumpctl is much more popular and provides great experience for developers. We did a big mistake when we were trying to preserve the old school coredumping in CWD. Regards, Jakub "___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org
