https://bugs.kde.org/show_bug.cgi?id=528308
Bug ID: 528308
Summary: baloo_file_extractor has no per-file limit: a file
that hangs a parser blocks content indexing and fills
swap on every login
Classification: Frameworks and Libraries
Product: frameworks-baloo
Version First 6.31.0
Reported In:
Platform: Arch Linux
OS: Linux
Status: REPORTED
Severity: normal
Priority: NOR
Component: Baloo File Daemon
Assignee: [email protected]
Reporter: [email protected]
Target Milestone: ---
Created attachment 198036
--> https://bugs.kde.org/attachment.cgi?id=198036&action=edit
Generator for a small PDF that hangs the extractor via Poppler (duplicate
/Kids, poppler#1836)
## Summary
If an extractor plugin never returns on a file, `baloo_file` waits for it
indefinitely. The file is never marked as failed, so the same thing happens
again each time the service starts. Content indexing stops for everything else,
and the extractor can use all available swap.
The trigger in my case is a Poppler bug
(https://gitlab.freedesktop.org/poppler/poppler/-/work_items/1836, with a
proposed fix in !2432), but this report is about Baloo's handling. Fixing
Poppler removes this trigger, not the problem: any parser hang or runaway
allocation in any extractor would have the same effect.
## Versions
- Baloo and KFileMetaData 6.31.0 (Arch Linux `baloo 6.31.0-2`, `kfilemetadata
6.31.0-1`)
- Plasma 6.8.2
- Qt 6.12.0
- Poppler 26.10.0
- Linux 7.2.11, Wayland, 46 GiB RAM, 24 GiB zram swap
## Steps to reproduce
**Warning:** this can use all of your swap. Stop it with `systemctl --user stop
kde-baloo.service`.
1. Generate the attached test case inside an indexed folder: `python3
gen_nested_kids.py 25 ~/Documents/nested-kids.pdf dup`
2. Let Baloo reach content indexing, or run `balooctl6 index
~/Documents/nested-kids.pdf`.
3. Watch `balooctl6 status` and the memory use of `baloo_file_extractor`.
4. Log out and back in, then repeat step 3.
I observed this with a similar existing file,
`qpdf/qtest/qpdf/shared-unnamed-field.pdf` from the qpdf 12.4.2 sources. I did
not run these exact steps with the generated file. If you're testing a Poppler
with !2432 applied, use `python3 gen_shared_parents.py 25
~/Documents/nested-kids.pdf` instead, which still hangs that build.
I have no backtrace from the live extractor, because `ptrace_scope=1` blocked
attaching. A `pdfinfo` run on the file shows the time going into
`FormField::FormField` and `Form::createFieldFromDict` in Poppler.
## Actual results
This repeated in every logged session from Oct 24 to Nov 2 2026:
- `balooctl6 status` kept showing the same 2.5 KB PDF as being indexed, with
6,791 files waiting.
- The service peaked at 11.8 GiB of memory, the `MemoryHigh=25%` limit, and
23.3 GiB of swap, which was all of it.
- The extractor wrote 0 bytes to disk in 6.5 hours, and the index file wasn't
modified after Oct 25.
- Its main thread stayed in state D, usually in
`__mem_cgroup_handle_over_high`.
- Being throttled at that limit also caused constant page-cache refaults. It
re-read about 985 GB of the index over 6.5 hours and used about 2 hours of CPU.
- Baloo logged nothing about the stall, and the file was never added to the
failed list.
Memory was never exhausted, so the OOM killer never ran. `MemoryHigh` slows the
process down, but nothing ends it.
## Expected results
Baloo should give up on the file, mark it as failed, and carry on with the rest
of the queue.
## Relevant code
In `src/file/filecontentindexer.cpp`, the indexer calls `process.index(idList)`
and then `loop.exec()`. That loop only ends on `ExtractorProcess::done` or
`failed`. `src/file/extractorprocess.cpp` only emits `failed` when the process
exits, so a process that never exits is never handled. I found no timeout,
watchdog or `setrlimit` in these files, including on master (`66936e40`). The
only timeout in `src/file/extractor/app.cpp` is for user idle detection.
## Possible fix
I haven't tested this. The recovery path already exists: when the extractor
exits abnormally, `failed` halves the batch until it reaches one file, then
calls `markFailed()`. A watchdog only has to end the extractor when it stops
making progress:
- Restart a timer on each `finishedIndexingFile`, and kill the extractor when
it expires. `QProcess::kill()` should report `CrashExit` and enter the existing
path.
- Base the limit on the extractor's CPU time rather than wall-clock time. The
service runs with `CPUWeight=1` and `IOWeight=1`, so a healthy extractor can
legitimately be slow on a busy machine.
- Also consider a memory limit per extractor, for example `RLIMIT_AS` or a
check on its RSS, so allocations don't have to fill swap before the timer
fires.
- A cgroup `MemoryMax` alone may not be enough. An OOM kill could stop the
whole service under systemd's default `OOMPolicy=stop`, so the file wouldn't
get marked as failed.
## Workaround
Add the folder to the exclusion list, for example `balooctl6 config add
excludeFolders ~/path`, and restart the service. On my machine that returned
swap use from 23 GiB to 0 within 25 seconds.
--
You are receiving this mail because:
You are watching all bug changes.
