Re: libksane seems to break QProcess::start calls
Hi Thiago and all, thanks a lot for the input so far! I'm really a bit lost here ... Am 04.03.22 um 01:06 schrieb Thiago Macieira: > Fix the problem where the problem is. I would really love to, esp. if this could be some upstream issue. And if not, I'd also love to fix it ;-) You can strace the process. Add the options -f -bexecve to see the current process and its forks & threads, but stop when it executes a child process. You should see SIGCHLD line, like this: I did it like this. I created one strace for the scanner not causing problems and scanned two pages. This resulted in 27,096 lines of output. Then I created one for the scanner causing the issue, also scanning two pages. This yielded a solid 270,807 lines :-O To be honest, I don't have a clue for what I search in those files ... would you be so kind to have a look? Or can I lessen the output, so this becomes more readable? I uploaded the traces here: https://l3u.de/tmp/strace_brscan.txt.xz https://l3u.de/tmp/strace_plustek.txt.xz Thanks again for all help! Cheers, Tobias
The KDE Patchset Collection has been rebased on top of Qt 5.15.3
The KDE Patchset Collection has been rebased on top of Qt 5.15.3 Some patches have been droped, some are still needed since we carry patches newer than 1 year old. Since this are rebases (to clearly show which patches are "ours", i.e. those on top of the v5.15.3-lts-lgpl tag) the kde/5.15 branches have been force-pushed to, don't get scared when fetching tells you that a force-update happened. Cheers, Albert
Re: libksane seems to break QProcess::start calls
On Thursday, 3 March 2022 11:00:07 PST Tobias Leupold wrote:
> Am Donnerstag, 3. März 2022, 19:06:21 CET schrieb Stefan Brüns:
> > Are you using either Qt5 < 5.15 or a kernel version which does not support
> > CLONE_FD? - then you are relying on SIGCHLD for process exit notification.
> >
> > CLONE_FD: https://lwn.net/Articles/636646/
> > Qt5: https://codereview.qt-project.org/c/qt/qtbase/+/108456/
> >
> > sane-backends/backend/plustek-usbhw.c messes with the signal handlers and
> > fails to restore it: `sigaction(..., ..., NULL)`
If CLONE_PIDFD is active, we don't use the SIGCHLD handler to be notified of
process exit, but SIGCHLD must not be ignored with SIG_IGN. That causes the
kernel not to notify anything.
https://gitlab.com/sane-project/backends/-/blob/master/backend/plustek-usbhw.c
I see sigaction, but it is only messing with SIGALRM. Yes, it should restore
previous the handler (probably SIG_DFL) after the alarm is done, but this
isn't the issue. It also messes with the signal block mask, but I don't see a
problem there either: it blocks and unblocks SIGALRM.
> I use Qt 5.15.2 and Gentoo's kernel 5.15.16. Is there something I can check
> to figure out if this could cause the problem? Apparently, there's no
> kernel option I can set?
You can strace the process. Add the options -f -bexecve to see the current
process and its forks & threads, but stop when it executes a child process.
You should see SIGCHLD line, like this:
kill(669903, SIGTERM) = 0
poll([{fd=3, events=0}, {fd=5, events=0}, {fd=6, events=POLLIN}, {fd=8,
events=POLLIN}, {fd=12, events=POLLIN}], 5, 5000) = 4 ([{fd=5,
revents=POLLERR}, {fd=6, revents=POLLHUP}, {fd=8, revents=POLLHUP}, {fd=12,
revents=POLLIN}])
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=669903, si_uid=1000,
si_status=SIGTERM, si_utime=0, si_stime=0} ---
fcntl(12, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
waitid(P_PIDFD, 12, {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=669903,
si_uid=1000, si_status=SIGTERM, si_utime=0, si_stime=0}, WNOHANG|WEXITED,
NULL) = 0
fd 12 is the pidfd in this case.
Anyway, search for SIGCHLD in the trace and see if anything is setting a
handler for it. On kernels >= 5.4, Qt doesn't do it, so if you see any mention
of it aside from deliveries like above, it came from elsewhere.
> Apart from that, the scanner in question actually does use the plustek
> backend! I fear there's no way to fix this in my code?!
Fix the problem where the problem is.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel DPG Cloud Engineering
KDE CI: Frameworks » kirigami » kf5-qt5 FreeBSDQt5.15 - Build # 792 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20FreeBSDQt5.15/792/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 22:57:53 + Build duration: 3 min 36 sec and counting JUnit Tests Name: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515 Failed: 16 test(s), Passed: 0 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_layers.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_pagepool.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_actiontoolbar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_avatar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_icon.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_keynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_listskeynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_mnemonicdata.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerouter.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_routerwindow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_theme.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_filterMouseEvents.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_invokables.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_onWheel.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_scrolling.qml
KDE CI: Frameworks » kcalendarcore » kf5-qt5 SUSEQt5.15 - Build # 201 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20SUSEQt5.15/201/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 21:51:28 + Build duration: 3 min 0 sec and counting BUILD ARTIFACTS acc/KF5CalendarCore-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report100% (2/2)94% (88/94)94% (88/94)74% (10764/14630)52% (5709/11052)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests100% (41/41)100% (41/41)97% (3770/3896)51% (1853/3638)src89% (47/53)89% (47/53)65% (6994/10734)52% (3856/7414)
KDE CI: Frameworks » kcalendarcore » kf5-qt5 FreeBSDQt5.15 - Build # 197 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20FreeBSDQt5.15/197/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 21:51:28 + Build duration: 1 min 50 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization
KDE CI: Frameworks » kirigami » kf5-qt5 FreeBSDQt5.15 - Build # 791 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20FreeBSDQt5.15/791/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 21:28:33 + Build duration: 42 sec and counting JUnit Tests Name: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515 Failed: 16 test(s), Passed: 0 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_layers.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_pagepool.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_actiontoolbar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_avatar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_icon.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_keynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_listskeynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_mnemonicdata.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerouter.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_routerwindow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_theme.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_filterMouseEvents.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_invokables.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_onWheel.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_scrolling.qml
KDE CI: Frameworks » kirigami » kf5-qt5 WindowsMSVCQt5.15 - Build # 688 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20WindowsMSVCQt5.15/688/ Project: kf5-qt5 WindowsMSVCQt5.15 Date of build: Thu, 03 Mar 2022 21:07:13 + Build duration: 1 min 49 sec and counting JUnit Tests Name: projectrootC_.CI.Job_Build Failed: 1 test(s), Passed: 15 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectrootC_.CI.Job_Build.autotests.pagepool/tst_layers.qml
KDE CI: Frameworks » kirigami » kf5-qt5 FreeBSDQt5.15 - Build # 790 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kirigami/job/kf5-qt5%20FreeBSDQt5.15/790/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 21:07:13 + Build duration: 1 min 10 sec and counting JUnit Tests Name: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515 Failed: 16 test(s), Passed: 0 test(s), Skipped: 0 test(s), Total: 16 test(s)Failed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_layers.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.pagepool/tst_pagepool.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_actiontoolbar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_avatar.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_icon.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_keynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_listskeynavigation.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_mnemonicdata.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerouter.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_pagerow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_routerwindow.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.tst_theme.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_filterMouseEvents.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_invokables.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_onWheel.qmlFailed: projectroot.usr.home.jenkins.workspace.Frameworks.kirigami.kf5-qt5_FreeBSDQt515.autotests.wheelhandler/tst_scrolling.qml
Re: libksane seems to break QProcess::start calls
Am Donnerstag, 3. März 2022, 19:06:21 CET schrieb Stefan Brüns: > Are you using either Qt5 < 5.15 or a kernel version which does not support > CLONE_FD? - then you are relying on SIGCHLD for process exit notification. > > CLONE_FD: https://lwn.net/Articles/636646/ > Qt5: https://codereview.qt-project.org/c/qt/qtbase/+/108456/ > > sane-backends/backend/plustek-usbhw.c messes with the signal handlers and > fails to restore it: `sigaction(..., ..., NULL)` I use Qt 5.15.2 and Gentoo's kernel 5.15.16. Is there something I can check to figure out if this could cause the problem? Apparently, there's no kernel option I can set? Apart from that, the scanner in question actually does use the plustek backend! I fear there's no way to fix this in my code?!
Re: libksane seems to break QProcess::start calls
On Donnerstag, 3. März 2022 13:54:32 CET Tobias Leupold wrote: > Hi all :-) > > I have a very odd problem, and I have no idea what could cause this or even > how to debug this. maybe, someone of you can give me a hint. > > I revently wrote a small helper program for one special purpose: Scanning > documents at a defined size, post processing them a bit and saving the > processed, compressed images as a PDF file to e.g. send them via mail. The > sources can be found at https://invent.kde.org/tleupold/scandoc/ . Are you using either Qt5 < 5.15 or a kernel version which does not support CLONE_FD? - then you are relying on SIGCHLD for process exit notification. CLONE_FD: https://lwn.net/Articles/636646/ Qt5: https://codereview.qt-project.org/c/qt/qtbase/+/108456/ sane-backends/backend/plustek-usbhw.c messes with the signal handlers and fails to restore it: `sigaction(..., ..., NULL)` Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen home: +49 241 53809034 mobile: +49 151 50412019 signature.asc Description: This is a digitally signed message part.
Re: libksane seems to break QProcess::start calls
Hi Klaas! > Interesting, I wrote a very similar little utility called PDFQuirk: > https://dragotin.github.io/quirksite/ Well, apparently, this is a quite common task to do ;-) > PDFQuirk has a very similar idea, yet it does not use libksane, but the > command line tool scanimage directly, started via QProcess. The idea > behind that is that the users should not need to tweak around with > scanner settings after the 'admin' set them, as in the average office, > the scanner does not change so often... I also thought about using scanimage (actually, scandoc is the GUI version of a shell script I used until now). But being able to set the scanner settings via the GUI (esp. the page size) seemed nicer to me. Once set, the settings are saved and restored per scanner, so that one only has to set them once, too. > As this only happens with one scanner and works fine with the other > (right?), Correct > I would point to the scanner driver that is loaded by sane. > Have you tried the scanimage command line utility from SANE if it works > with the scanner in question? Using scanimage on the console works without a problem. The thing I really don't get is how scanning two images can ever affect a QProcess started somewhere else ... I mean the convert process does work, but QProcess thinks that it doesn't finish ... but only after two scans ...
KDE CI: Frameworks » kwindowsystem » kf5-qt5 FreeBSDQt5.15 - Build # 169 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kwindowsystem/job/kf5-qt5%20FreeBSDQt5.15/169/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:09:55 + Build duration: 1 min 15 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 12 test(s), Skipped: 0 test(s), Total: 13 test(s)Failed: projectroot.autotests.kwindowsystem_kwindowsystemplatformwaylandtest
KDE CI: Frameworks » kcrash » kf5-qt5 FreeBSDQt5.15 - Build # 124 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcrash/job/kf5-qt5%20FreeBSDQt5.15/124/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:10:23 + Build duration: 32 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 2 test(s), Skipped: 0 test(s), Total: 3 test(s)Failed: projectroot.autotests.metadatatest
KDE CI: Frameworks » kcrash » kf5-qt5 FreeBSDQt5.15 - Build # 123 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcrash/job/kf5-qt5%20FreeBSDQt5.15/123/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:09:51 + Build duration: 31 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 2 test(s), Skipped: 0 test(s), Total: 3 test(s)Failed: projectroot.autotests.metadatatest
KDE CI: Frameworks » kwindowsystem » kf5-qt5 FreeBSDQt5.15 - Build # 168 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kwindowsystem/job/kf5-qt5%20FreeBSDQt5.15/168/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 17:07:31 + Build duration: 2 min 22 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 12 test(s), Skipped: 0 test(s), Total: 13 test(s)Failed: projectroot.autotests.kwindowsystem_kwindowsystemplatformwaylandtest
Re: libksane seems to break QProcess::start calls
Am 03.03.22 um 13:54 schrieb Tobias Leupold:
Hi Tobias,
I have a very odd problem, and I have no idea what could cause this or even
how to debug this. maybe, someone of you can give me a hint.
I revently wrote a small helper program for one special purpose: Scanning
documents at a defined size, post processing them a bit and saving the
processed, compressed images as a PDF file to e.g. send them via mail. The
sources can be found at https://invent.kde.org/tleupold/scandoc/ .
Interesting, I wrote a very similar little utility called PDFQuirk:
https://dragotin.github.io/quirksite/
It uses libksane, ImageMagick's convert and pdfjam as helper programs. This
may be too special or too hacky to become e.g. an official KDE extragear
program, but that's another thing.
PDFQuirk has a very similar idea, yet it does not use libksane, but the
command line tool scanimage directly, started via QProcess. The idea
behind that is that the users should not need to tweak around with
scanner settings after the 'admin' set them, as in the average office,
the scanner does not change so often...
However the problem is:
As said, the program uses convert to post process the scanned images. I use
QProcess to run the respective command, in a procedural, synchronous way, as
the command is typically finished within fractions of a second. The call
strips down to:
QProcess process;
process.start(command, arguments);
waitForFinished();
Using one scanner I have (it's a Brother MFC device), this works without a
problem. Everything is fine. But using another one, a CanoScan LiDE 25, a
really strange problem happens:
As this only happens with one scanner and works fine with the other
(right?), I would point to the scanner driver that is loaded by sane.
Have you tried the scanimage command line utility from SANE if it works
with the scanner in question?
regards,
Klaas
After having scanned the first page, everything works as expected. But after
having scanned the second one, the QProcess call doesn't exit anymore. It runs
normally, and the output file is created. But it doesn't return, until it's
killed by the waitForFinished() call. ps lists the process as "defunct".
As expected, the GUI freezes for 30 seconds (the default timeout). But after
the process is killed, the GUI is still frozen for another 30 seconds (why?!),
then it becomes responsive again and the post processed image is added like if
the call would have exited normally.
It's also not about the "convert" call. Each and every QProcess I start after
the seconds scan does not exit anymore. Even something like "dmesg" or such.
After the first scan, everything is fine, after the second scan,
QProcess::start does not exit anymore. As long as I don't do a second scan, I
can start as many QProcess processes as I want, and all exit normally. But not
anymore after the second page.
I also tried to create the QProcess on the heap, and to implement the command
run asynchronously. The result is the same: After the first scan, the process
returns normally, after the second scan, it doesn't exit anymore ("can't
start, already running").
To make it even more peculiar: At first, I implemented the convert process to
read from stdin and write to stdout, piped the image data to it, and read the
output to get the processed image directly. This caused no problem, no matter
how much scans I did. But later on, I needed to call programs not reading from
stdin.
So ... how can that even happen? Where do the 30 seconds unresponsiveness come
from, after the QProcess has already been killed?! Is this something that
libksane causes? How can it influence a completely unrealted QProcess call? Or
did I simply write crappy code?!
If anybody has any idea about this, I would really appreciate some
enlightenment ;-)
Thanks for all help in advance!
Cheers, Tobias
libksane seems to break QProcess::start calls
Hi all :-)
I have a very odd problem, and I have no idea what could cause this or even
how to debug this. maybe, someone of you can give me a hint.
I revently wrote a small helper program for one special purpose: Scanning
documents at a defined size, post processing them a bit and saving the
processed, compressed images as a PDF file to e.g. send them via mail. The
sources can be found at https://invent.kde.org/tleupold/scandoc/ .
It uses libksane, ImageMagick's convert and pdfjam as helper programs. This
may be too special or too hacky to become e.g. an official KDE extragear
program, but that's another thing.
However the problem is:
As said, the program uses convert to post process the scanned images. I use
QProcess to run the respective command, in a procedural, synchronous way, as
the command is typically finished within fractions of a second. The call
strips down to:
QProcess process;
process.start(command, arguments);
waitForFinished();
Using one scanner I have (it's a Brother MFC device), this works without a
problem. Everything is fine. But using another one, a CanoScan LiDE 25, a
really strange problem happens:
After having scanned the first page, everything works as expected. But after
having scanned the second one, the QProcess call doesn't exit anymore. It runs
normally, and the output file is created. But it doesn't return, until it's
killed by the waitForFinished() call. ps lists the process as "defunct".
As expected, the GUI freezes for 30 seconds (the default timeout). But after
the process is killed, the GUI is still frozen for another 30 seconds (why?!),
then it becomes responsive again and the post processed image is added like if
the call would have exited normally.
It's also not about the "convert" call. Each and every QProcess I start after
the seconds scan does not exit anymore. Even something like "dmesg" or such.
After the first scan, everything is fine, after the second scan,
QProcess::start does not exit anymore. As long as I don't do a second scan, I
can start as many QProcess processes as I want, and all exit normally. But not
anymore after the second page.
I also tried to create the QProcess on the heap, and to implement the command
run asynchronously. The result is the same: After the first scan, the process
returns normally, after the second scan, it doesn't exit anymore ("can't
start, already running").
To make it even more peculiar: At first, I implemented the convert process to
read from stdin and write to stdout, piped the image data to it, and read the
output to get the processed image directly. This caused no problem, no matter
how much scans I did. But later on, I needed to call programs not reading from
stdin.
So ... how can that even happen? Where do the 30 seconds unresponsiveness come
from, after the QProcess has already been killed?! Is this something that
libksane causes? How can it influence a completely unrealted QProcess call? Or
did I simply write crappy code?!
If anybody has any idea about this, I would really appreciate some
enlightenment ;-)
Thanks for all help in advance!
Cheers, Tobias
Re: Critical Denial of Service bugs in Discover
I'd say wireshark is too low level for what the problem is here. We are talking about having too many HTTP requests for specific URLs. I can think two main measures: - Trigger an alarm (an e-mail notification?) if there's a specific UserAgent that has a specific portion of the queries we have in a specific day in the services we care about. - Offer plots to see how queries by UserAgent evolve over the last couple of months (or couple of years). Aleix On Thu, Mar 3, 2022 at 9:59 AM Ben Cooksley wrote: > On Thu, Mar 3, 2022 at 8:41 AM Aleix Pol wrote: > >> (dropping the distros list) >> >> @sysadmin have you been able to look into any tools we devs can have to >> make sure this situation doesn't repeat in the future? >> > > Hi Aleix, > > To be honest i've been struggling to think of ways that we could detect > this on the server side prior to it becoming a massive issue. > By the time an issue is evident server side it is usually much too late. > > The main tools i'd usually recommend would be the standard tools you would > use for monitoring the network activity of any application - such as > Wireshark. > > Is there something you were thinking of specifically in terms of us being > able to provide? > > Thanks, > Ben > > >> >> Aleix >> >> On Thu, Feb 10, 2022 at 1:10 PM Aleix Pol wrote: >> >>> On Thu, Feb 10, 2022 at 11:05 AM Ben Cooksley wrote: >>> > >>> > >>> > >>> > On Thu, Feb 10, 2022 at 8:20 AM Aleix Pol wrote: >>> >> >>> >> [Snip] >>> >> >>> >> We still haven't discussed here is how to prevent this problem from >>> >> happening again. >>> >> >>> >> If we don't have information about what is happening, we cannot fix >>> problems. >>> > >>> > >>> > Part of the issue here is that the problem only came to Sysadmin >>> attention very recently, when the system ran out of disk space as a result >>> of growing log files. >>> > It was at that point we realised we had a serious problem. >>> > >>> > Prior to that the system load hadn't climbed to dangerous levels (> >>> number of CPU cores) and Apache was keeping up with the traffic, so none of >>> our other monitoring was tripped. >>> > >>> > If you have any thoughts on what sort of information you are thinking >>> of that would be helpful. >>> >>> We could have plots of the amount of queries we get with a KNewStuff/* >>> user-agent over time and their distribution. >>> >>> > It would definitely be helpful though to know when new software is >>> going to be released that will be interacting with the servers as we will >>> then be able to monitor for abnormalities. >>> >>> We make big announcements of every Plasma release... (?) >>> >>> >> Is there anything that could be done in this front? The issue here >>> >> could have been addressed months ago, we just never knew it was >>> >> happening. >>> > >>> > >>> > One possibility that did occur to me today would be for us to >>> integrate some kind of killswitch that our applications would check on >>> first initialisation of functionality that talks to KDE.org servers. >>> > This would allow us to disable the functionality in question on user >>> systems. >>> > >>> > The check would only be done on first initialization to keep load low, >>> while still ensuring all users eventually are affected by the killswitch >>> (as they will eventually need to logout/reboot for some reason or another). >>> > >>> > The killswitch would probably work best if it had some kind of version >>> check in it so we could specify which versions are disabled. >>> > That would allow for subsequent updates - once delivered by >>> distributions - to restore the functionality (while leaving it disabled for >>> those who haven't updated). >>> >>> The file we are serving here effectively is the kill switch to all of >>> KNewStuff. >>> >>> Aleix >>> >>
KDE CI: Frameworks » kcalendarcore » kf5-qt5 WindowsMSVCQt5.15 - Build # 184 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20WindowsMSVCQt5.15/184/ Project: kf5-qt5 WindowsMSVCQt5.15 Date of build: Thu, 03 Mar 2022 11:10:41 + Build duration: 34 min and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 493 test(s), Skipped: 0 test(s), Total: 494 test(s)Failed: projectroot.autotests.testdateserialization
KDE CI: Frameworks » kcalendarcore » kf5-qt5 FreeBSDQt5.15 - Build # 196 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20FreeBSDQt5.15/196/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:18:02 + Build duration: 6 min 7 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization
KDE CI: Frameworks » kio » kf5-qt5 FreeBSDQt5.15 - Build # 1432 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20FreeBSDQt5.15/1432/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:14:51 + Build duration: 7 min 40 sec and counting JUnit Tests Name: projectroot Failed: 3 test(s), Passed: 57 test(s), Skipped: 0 test(s), Total: 60 test(s)Failed: projectroot.autotests.kiocore_ktcpsockettestFailed: projectroot.autotests.kiofilewidgets_kfilewidgettestFailed: projectroot.autotests.kiowidgets_kdirlistertestName: projectroot.autotests Failed: 0 test(s), Passed: 6 test(s), Skipped: 0 test(s), Total: 6 test(s)Name: projectroot.src.ioslaves.trash Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot.src.kpasswdserver Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)
KDE CI: Frameworks » kio » kf5-qt5 SUSEQt5.15 - Build # 1415 - Fixed!
BUILD SUCCESS Build URL https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20SUSEQt5.15/1415/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 11:10:14 + Build duration: 11 min and counting BUILD ARTIFACTS acc/KF5KIO-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 0 test(s), Passed: 61 test(s), Skipped: 0 test(s), Total: 61 test(s)Name: projectroot.autotests Failed: 0 test(s), Passed: 6 test(s), Skipped: 0 test(s), Total: 6 test(s)Name: projectroot.src.ioslaves.trash Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot.src.kpasswdserver Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s) Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report61% (25/41)69% (298/429)69% (298/429)58% (39323/67995)42% (21770/52050)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests95% (61/64)95% (61/64)91% (11408/12474)46% (7159/15602)autotests.http100% (5/5)100% (5/5)99% (527/528)58% (167/290)autotests.kcookiejar100% (1/1)100% (1/1)94% (173/185)63% (70/112)src100% (1/1)100% (1/1)89% (8/9)71% (10/14)src.core88% (107/121)88% (107/121)61% (9367/15288)52% (4629/8876)src.core.kssl100% (1/1)100% (1/1)38% (33/86)50% (2/4)src.filewidgets79% (31/39)79% (31/39)58% (5405/9396)44% (2324/5325)src.gui100% (11/11)100% (11/11)73% (867/1188)58% (458/790)src.gui.systemd50% (2/4)50% (2/4)4% (7/178)1% (1/108)src.ioslaves.file100% (7/7)100% (7/7)56% (785/1400)42% (518/1227)src.ioslaves.file.kauth0% (0/2)0% (0/2)0% (0/187)0% (0/83)src.ioslaves.ftp100% (2/2)100% (2/2)40% (551/1380)30% (435/1430)src.ioslaves.help0% (0/5)0% (0/5)0% (0/253)0% (0/138)src.ioslaves.http88% (7/8)88% (7/8)43% (1873/4354)37% (1359/3719)src.ioslaves.http.kcookiejar40% (2/5)40% (2/5)49% (664/1364)56% (588/1053)src.ioslaves.remote100% (2/2)100% (2/2)25%
KDE CI: Frameworks » kcalendarcore » kf5-qt5 SUSEQt5.15 - Build # 200 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20SUSEQt5.15/200/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 11:10:41 + Build duration: 7 min 1 sec and counting BUILD ARTIFACTS acc/KF5CalendarCore-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report100% (2/2)94% (88/94)94% (88/94)73% (10721/14587)52% (5661/10988)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests100% (41/41)100% (41/41)97% (3729/3855)51% (1836/3604)src89% (47/53)89% (47/53)65% (6992/10732)52% (3825/7384)
KDE CI: Frameworks » kcalendarcore » kf5-qt5 FreeBSDQt5.15 - Build # 195 - Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kcalendarcore/job/kf5-qt5%20FreeBSDQt5.15/195/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:10:41 + Build duration: 7 min 4 sec and counting JUnit Tests Name: projectroot Failed: 1 test(s), Passed: 494 test(s), Skipped: 0 test(s), Total: 495 test(s)Failed: projectroot.autotests.testdateserialization
KDE CI: Frameworks » kio » kf5-qt5 FreeBSDQt5.15 - Build # 1431 - Still Unstable!
BUILD UNSTABLE Build URL https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20FreeBSDQt5.15/1431/ Project: kf5-qt5 FreeBSDQt5.15 Date of build: Thu, 03 Mar 2022 11:09:30 + Build duration: 5 min 13 sec and counting JUnit Tests Name: projectroot Failed: 4 test(s), Passed: 56 test(s), Skipped: 0 test(s), Total: 60 test(s)Failed: projectroot.autotests.kiocore_jobtestFailed: projectroot.autotests.kiocore_ktcpsockettestFailed: projectroot.autotests.kiowidgets_kdirlistertestFailed: projectroot.autotests.kiowidgets_kdirmodeltestName: projectroot.autotests Failed: 0 test(s), Passed: 6 test(s), Skipped: 0 test(s), Total: 6 test(s)Name: projectroot.src.ioslaves.trash Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot.src.kpasswdserver Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)
KDE CI: Frameworks » kholidays » kf5-qt5 SUSEQt5.15 - Build # 137 - Fixed!
BUILD SUCCESS Build URL https://build.kde.org/job/Frameworks/job/kholidays/job/kf5-qt5%20SUSEQt5.15/137/ Project: kf5-qt5 SUSEQt5.15 Date of build: Thu, 03 Mar 2022 11:10:14 + Build duration: 2 min 34 sec and counting BUILD ARTIFACTS acc/KF5Holidays-5.92.0.xml JUnit Tests Name: (root) Failed: 0 test(s), Passed: 1 test(s), Skipped: 0 test(s), Total: 1 test(s)Name: projectroot Failed: 0 test(s), Passed: 5 test(s), Skipped: 0 test(s), Total: 5 test(s) Cobertura Report Project Coverage Summary Name PackagesFilesClassesLinesConditionalsCobertura Coverage Report80% (4/5)88% (15/17)88% (15/17)68% (1585/2327)49% (774/1594)Coverage Breakdown by Package Name FilesClassesLinesConditionalsautotests100% (5/5)100% (5/5)99% (385/387)57% (125/218)src100% (7/7)100% (7/7)71% (582/824)48% (336/701)src.declarative0% (0/2)0% (0/2)0% (0/28)0% (0/8)src.parsers100% (2/2)100% (2/2)43% (316/727)34% (142/414)src.parsers.plan2100% (1/1)100% (1/1)84% (302/361)68% (171/253)
KDE CI: Frameworks » kholidays » kf5-qt5 SUSEQt5.15 - Build # 136 - Failure!
BUILD FAILURE
Build URL
https://build.kde.org/job/Frameworks/job/kholidays/job/kf5-qt5%20SUSEQt5.15/136/
Project:
kf5-qt5 SUSEQt5.15
Date of build:
Thu, 03 Mar 2022 11:09:40 +
Build duration:
32 sec and counting
CONSOLE OUTPUT
Started by an SCM changeRunning in Durability level: PERFORMANCE_OPTIMIZED[Pipeline] Start of Pipeline[Pipeline] nodeStill waiting to schedule taskWaiting for next available executor on ‘SUSEQt5.15’Running on Docker Swarm-04a148b81353 in /home/jenkins/workspace/Frameworks/kholidays/kf5-qt5 SUSEQt5.15[Pipeline] {[Pipeline] timestamps[Pipeline] {[Pipeline] catchError[Pipeline] {[Pipeline] stage[Pipeline] { (Checkout Sources)[Pipeline] checkout[2022-03-03T11:10:05.055Z] The recommended git tool is: NONE[2022-03-03T11:10:06.818Z] No credentials specified[2022-03-03T11:10:06.824Z] Cloning the remote Git repository[2022-03-03T11:10:06.856Z] Cloning repository https://invent.kde.org/frameworks/kholidays.git[2022-03-03T11:10:06.933Z] > git init /home/jenkins/workspace/Frameworks/kholidays/kf5-qt5 SUSEQt5.15 # timeout=10[2022-03-03T11:10:06.978Z] Fetching upstream changes from https://invent.kde.org/frameworks/kholidays.git[2022-03-03T11:10:06.978Z] > git --version # timeout=10[2022-03-03T11:10:06.990Z] > git --version # 'git version 2.35.1'[2022-03-03T11:10:06.991Z] > git fetch --tags --force --progress -- https://invent.kde.org/frameworks/kholidays.git +refs/heads/*:refs/remotes/origin/* # timeout=120[2022-03-03T11:10:08.700Z] Avoid second fetch[2022-03-03T11:10:08.730Z] Checking out Revision bbe08c4c7bd569841fcf04d3d727fc97053a1a07 (origin/master)[2022-03-03T11:10:08.683Z] > git config remote.origin.url https://invent.kde.org/frameworks/kholidays.git # timeout=10[2022-03-03T11:10:08.689Z] > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10[2022-03-03T11:10:08.705Z] > git rev-parse origin/master^{commit} # timeout=10[2022-03-03T11:10:08.738Z] > git config core.sparsecheckout # timeout=10[2022-03-03T11:10:08.754Z] > git checkout -f bbe08c4c7bd569841fcf04d3d727fc97053a1a07 # timeout=10[2022-03-03T11:10:12.786Z] Commit message: "Add Qt6 Android CI"[2022-03-03T11:10:12.792Z] > git rev-list --no-walk c584c3ae5aea63d908f066554e63d405937b5919 # timeout=10[Pipeline] checkout[2022-03-03T11:10:12.859Z] The recommended git tool is: NONE[2022-03-03T11:10:12.872Z] No credentials specified[2022-03-03T11:10:12.879Z] Cloning the remote Git repository[2022-03-03T11:10:13.466Z] ERROR: Error cloning remote repo 'origin'[2022-03-03T11:10:13.466Z] hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://invent.kde.org/sysadmin/ci-tooling.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:[2022-03-03T11:10:13.466Z] stdout: [2022-03-03T11:10:13.466Z] stderr: error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500[2022-03-03T11:10:13.466Z] fatal: expected flush after ref listing[2022-03-03T11:10:13.466Z] [2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2450)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2051)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:573)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:802)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)[2022-03-03T11:10:13.466Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)[2022-03-03T11:10:13.466Z] at hudson.remoting.UserRequest.perform(UserRequest.java:211)[2022-03-03T11:10:13.466Z] at hudson.remoting.UserRequest.perform(UserRequest.java:54)[2022-03-03T11:10:13.466Z] at hudson.remoting.Request$2.run(Request.java:375)[2022-03-03T11:10:13.466Z] at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)[2022-03-03T11:10:13.466Z] at java.util.concurrent.FutureTask.run(FutureTask.java:264)[2022-03-03T11:10:13.466Z] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)[2022-03-03T11:10:13.466Z] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)[2022-03-03T11:10:13.466Z] at java.lang.Thread.run(Thread.java:829)[2022-03-03T11:10:13.467Z] Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to Docker Swarm-04a148b81353[2022-03-03T11:10:13.467Z] at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)[2022-03-03T11:10:13.467Z] at
KDE CI: Frameworks » kio » kf5-qt5 SUSEQt5.15 - Build # 1414 - Failure!
BUILD FAILURE
Build URL
https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20SUSEQt5.15/1414/
Project:
kf5-qt5 SUSEQt5.15
Date of build:
Thu, 03 Mar 2022 11:09:30 +
Build duration:
43 sec and counting
CONSOLE OUTPUT
Started by an SCM changeRunning in Durability level: PERFORMANCE_OPTIMIZED[Pipeline] Start of Pipeline[Pipeline] nodeStill waiting to schedule taskWaiting for next available executor on ‘SUSEQt5.15’Running on Docker Swarm-d71eaea88533 in /home/jenkins/workspace/Frameworks/kio/kf5-qt5 SUSEQt5.15[Pipeline] {[Pipeline] timestamps[Pipeline] {[Pipeline] catchError[Pipeline] {[Pipeline] stage[Pipeline] { (Checkout Sources)[Pipeline] checkout[2022-03-03T11:09:52.439Z] The recommended git tool is: NONE[2022-03-03T11:09:53.484Z] No credentials specified[2022-03-03T11:09:53.487Z] Cloning the remote Git repository[2022-03-03T11:09:53.501Z] Cloning repository https://invent.kde.org/frameworks/kio.git[2022-03-03T11:09:53.556Z] > git init /home/jenkins/workspace/Frameworks/kio/kf5-qt5 SUSEQt5.15 # timeout=10[2022-03-03T11:09:53.580Z] Fetching upstream changes from https://invent.kde.org/frameworks/kio.git[2022-03-03T11:09:53.581Z] > git --version # timeout=10[2022-03-03T11:09:53.595Z] > git --version # 'git version 2.35.1'[2022-03-03T11:09:53.596Z] > git fetch --tags --force --progress -- https://invent.kde.org/frameworks/kio.git +refs/heads/*:refs/remotes/origin/* # timeout=120[2022-03-03T11:10:07.891Z] Avoid second fetch[2022-03-03T11:10:07.921Z] Checking out Revision fa08838141f18959e545f08c9fcc41cffd447555 (origin/master)[2022-03-03T11:10:07.852Z] > git config remote.origin.url https://invent.kde.org/frameworks/kio.git # timeout=10[2022-03-03T11:10:07.872Z] > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10[2022-03-03T11:10:07.896Z] > git rev-parse origin/master^{commit} # timeout=10[2022-03-03T11:10:07.928Z] > git config core.sparsecheckout # timeout=10[2022-03-03T11:10:07.934Z] > git checkout -f fa08838141f18959e545f08c9fcc41cffd447555 # timeout=10[2022-03-03T11:10:11.597Z] Commit message: "Add Qt6 Android CI"[2022-03-03T11:10:11.601Z] > git rev-list --no-walk 2ffcc2cab351b8d64682ccfb7f2c0c1cfa81edbe # timeout=10[Pipeline] checkout[2022-03-03T11:10:11.749Z] The recommended git tool is: NONE[2022-03-03T11:10:11.757Z] No credentials specified[2022-03-03T11:10:11.760Z] Cloning the remote Git repository[2022-03-03T11:10:11.762Z] Cloning repository https://invent.kde.org/sysadmin/ci-tooling.git[2022-03-03T11:10:11.762Z] > git init /home/jenkins/workspace/Frameworks/kio/kf5-qt5 SUSEQt5.15/ci-tooling # timeout=10[2022-03-03T11:10:11.770Z] Fetching upstream changes from https://invent.kde.org/sysadmin/ci-tooling.git[2022-03-03T11:10:11.771Z] > git --version # timeout=10[2022-03-03T11:10:11.783Z] > git --version # 'git version 2.35.1'[2022-03-03T11:10:11.784Z] > git fetch --tags --force --progress -- https://invent.kde.org/sysadmin/ci-tooling.git +refs/heads/*:refs/remotes/origin/* # timeout=10[2022-03-03T11:10:13.115Z] Avoid second fetch[2022-03-03T11:10:13.126Z] Checking out Revision b4fb37c3190807b2b3223e4967638477f54f70bc (origin/master)[2022-03-03T11:10:13.153Z] Commit message: "Initial Android Qt6/KF6 CI image"[Pipeline] checkout[2022-03-03T11:10:13.160Z] The recommended git tool is: NONE[2022-03-03T11:10:13.166Z] No credentials specified[2022-03-03T11:10:13.169Z] Cloning the remote Git repository[2022-03-03T11:10:13.493Z] ERROR: Error cloning remote repo 'origin'[2022-03-03T11:10:13.493Z] hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://invent.kde.org/sysadmin/repo-metadata.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:[2022-03-03T11:10:13.493Z] stdout: [2022-03-03T11:10:13.493Z] stderr: fatal: unable to access 'https://invent.kde.org/sysadmin/repo-metadata.git/': The requested URL returned error: 500[2022-03-03T11:10:13.493Z] [2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2450)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2051)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:573)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:802)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:161)[2022-03-03T11:10:13.493Z] at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$GitCommandMasterToSlaveCallable.call(RemoteGitImpl.java:154)[2022-03-03T11:10:13.493Z] at hudson.remoting.UserRequest.perform(UserRequest.java:211)[2022-03-03T11:10:13.493Z] at
Re: Critical Denial of Service bugs in Discover
On Thu, Mar 3, 2022 at 8:41 AM Aleix Pol wrote: > (dropping the distros list) > > @sysadmin have you been able to look into any tools we devs can have to > make sure this situation doesn't repeat in the future? > Hi Aleix, To be honest i've been struggling to think of ways that we could detect this on the server side prior to it becoming a massive issue. By the time an issue is evident server side it is usually much too late. The main tools i'd usually recommend would be the standard tools you would use for monitoring the network activity of any application - such as Wireshark. Is there something you were thinking of specifically in terms of us being able to provide? Thanks, Ben > > Aleix > > On Thu, Feb 10, 2022 at 1:10 PM Aleix Pol wrote: > >> On Thu, Feb 10, 2022 at 11:05 AM Ben Cooksley wrote: >> > >> > >> > >> > On Thu, Feb 10, 2022 at 8:20 AM Aleix Pol wrote: >> >> >> >> [Snip] >> >> >> >> We still haven't discussed here is how to prevent this problem from >> >> happening again. >> >> >> >> If we don't have information about what is happening, we cannot fix >> problems. >> > >> > >> > Part of the issue here is that the problem only came to Sysadmin >> attention very recently, when the system ran out of disk space as a result >> of growing log files. >> > It was at that point we realised we had a serious problem. >> > >> > Prior to that the system load hadn't climbed to dangerous levels (> >> number of CPU cores) and Apache was keeping up with the traffic, so none of >> our other monitoring was tripped. >> > >> > If you have any thoughts on what sort of information you are thinking >> of that would be helpful. >> >> We could have plots of the amount of queries we get with a KNewStuff/* >> user-agent over time and their distribution. >> >> > It would definitely be helpful though to know when new software is >> going to be released that will be interacting with the servers as we will >> then be able to monitor for abnormalities. >> >> We make big announcements of every Plasma release... (?) >> >> >> Is there anything that could be done in this front? The issue here >> >> could have been addressed months ago, we just never knew it was >> >> happening. >> > >> > >> > One possibility that did occur to me today would be for us to integrate >> some kind of killswitch that our applications would check on first >> initialisation of functionality that talks to KDE.org servers. >> > This would allow us to disable the functionality in question on user >> systems. >> > >> > The check would only be done on first initialization to keep load low, >> while still ensuring all users eventually are affected by the killswitch >> (as they will eventually need to logout/reboot for some reason or another). >> > >> > The killswitch would probably work best if it had some kind of version >> check in it so we could specify which versions are disabled. >> > That would allow for subsequent updates - once delivered by >> distributions - to restore the functionality (while leaving it disabled for >> those who haven't updated). >> >> The file we are serving here effectively is the kill switch to all of >> KNewStuff. >> >> Aleix >> >
