[frameworks-baloo] [Bug 392877] balooctl index command with large number of files corrupts db

[Nate Graham]

https://bugs.kde.org/show_bug.cgi?id=392877

Nate Graham  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 Status|REPORTED|RESOLVED

--- Comment #5 from Nate Graham  ---
Cool.

-- 
You are receiving this mail because:
You are watching all bug changes.

[frameworks-baloo] [Bug 392877] balooctl index command with large number of files corrupts db

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=392877

tagwer...@innerjoin.org changed:

   What|Removed |Added

 CC||tagwer...@innerjoin.org

--- Comment #4 from tagwer...@innerjoin.org ---
(In reply to Nate Graham from comment #3)
> Is this 100% fixed now? Or is there still anything left to do?
Checked, on an index of 2,000,000 files (recorded Go games), the balooctl
option to clear the entries for 1 files:

balooctl clear 2016-01*

and reindex them

balooctl index 2016-01*

in one transaction. Done on a system with constrained RAM so that transaction
filled the RAM and extended into swap. It completed; the index was seemingly
OK.

Can never be sure, but good enough?

-- 
You are receiving this mail because:
You are watching all bug changes.

[frameworks-baloo] [Bug 392877] balooctl index command with large number of files corrupts db

[Nate Graham]

https://bugs.kde.org/show_bug.cgi?id=392877

Nate Graham  changed:

   What|Removed |Added

  Component|Engine  |balooctl
 CC||n...@kde.org

--- Comment #3 from Nate Graham  ---
Is this 100% fixed now? Or is there still anything left to do?

-- 
You are receiving this mail because:
You are watching all bug changes.

[frameworks-baloo] [Bug 392877] balooctl index command with large number of files corrupts db

[Stefan Brüns]

https://bugs.kde.org/show_bug.cgi?id=392877

--- Comment #2 from Stefan Brüns  ---
Git commit e1d1b7e87ff1e8ce6a7e03ecdf2902322cb8624a by Stefan Brüns.
Committed on 29/05/2018 at 23:47.
Pushed by bruns into branch 'master'.

Avoid crash when reading corrupt data from document terms db

Summary:
The terms db contains terms, where each terms is stored independently
(terminated with 0), or as a suffix to the previous term (terminated with
1).
In case of corrupted data, the first terminator seen may be a 1, which
leads to a crash when trying to access the previous term with
QVector<>::last().
Show a debug message, to give a hint about the bad data, which can be
fixed by reindexing the relevant file.
Related: bug 392878

Test Plan:
Corrupt the database
Run balooshow -x 

Reviewers: #baloo, michaelh, ngraham, #frameworks, dhaumann

Reviewed By: dhaumann

Subscribers: dhaumann, kde-frameworks-devel, #frameworks

Tags: #frameworks, #baloo

Differential Revision: https://phabricator.kde.org/D12047

M  +5-0src/codecs/doctermscodec.cpp
M  +5-1src/engine/documentdb.cpp

https://commits.kde.org/baloo/e1d1b7e87ff1e8ce6a7e03ecdf2902322cb8624a

-- 
You are receiving this mail because:
You are watching all bug changes.

[frameworks-baloo] [Bug 392877] balooctl index command with large number of files corrupts db

[Michael Heidelbach]

https://bugs.kde.org/show_bug.cgi?id=392877

Michael Heidelbach  changed:

   What|Removed |Added

 CC||ottw...@gmail.com

--- Comment #1 from Michael Heidelbach  ---
(In reply to Stefan Brüns from comment #0)
> When running "balooctl index /*", (at least) the contents of the
> terms db are garbage afterwards.
I could not reproduce. Maybe I overlooked the "garbage", could you specify
please.
balooctl index /*" => balooshow -x b.epub >multi.txt
balooctl disable + enable
for f in /*; do balooctl index "$f" ; done => balooshow -x b.epub
>single.txt

diff multi.txt single.txt 
14d13
< title: buddenbrooks einer familie verfall
15a15
> title: buddenbrooks einer familie verfall

-- 
You are receiving this mail because:
You are watching all bug changes.

[frameworks-baloo] [Bug 392877] balooctl index command with large number of files corrupts db

[Stefan Brüns]

https://bugs.kde.org/show_bug.cgi?id=392877

Stefan Brüns  changed:

   What|Removed |Added

   See Also||https://bugs.kde.org/show_b
   ||ug.cgi?id=392878

-- 
You are receiving this mail because:
You are watching all bug changes.