[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Daniel Vrátil via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=341884

Daniel Vrátil  changed:

   What|Removed |Added

 CC||jprofeso...@o2.pl

--- Comment #15 from Daniel Vrátil  ---
*** Bug 344937 has been marked as a duplicate of this bug. ***

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Scarlett Clark]

https://bugs.kde.org/show_bug.cgi?id=341884

Scarlett Clark  changed:

   What|Removed |Added

 CC||sgcl...@kubuntu.org

--- Comment #14 from Scarlett Clark  ---
I am working on the *ubuntu family packaging for this patch today.
Scarlett

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[graham]

https://bugs.kde.org/show_bug.cgi?id=341884

gra...@howlingfrog.com changed:

   What|Removed |Added

 CC||gra...@howlingfrog.com

--- Comment #13 from gra...@howlingfrog.com ---
Daniel, *THANK YOU* for fixing this.

I only _just_ installed the fix on my Fedora-21 machine, and after running
akonadictl fsck saw that it moved out just shy of 90k files from my
file_db_data directory (leaving me with just under 5k left).  And that's just
what's accumulated since I last ran akonadictl fsck back on June 30th.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #12 from Martin Steigerwald m...@teamix.de ---
Thanks a lot, Daniel. I notified Debian Qt/KDE team about it.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Daniel Vrátil]

https://bugs.kde.org/show_bug.cgi?id=341884

Daniel Vrátil dvra...@kde.org changed:

   What|Removed |Added

 Status|CONFIRMED   |RESOLVED
 CC||dvra...@kde.org
  Latest Commit||http://commits.kde.org/akon
   ||adi/9c0dc6b3f0826d32eac310b
   ||2e7ecd858ca3df681
 Resolution|--- |FIXED

--- Comment #11 from Daniel Vrátil dvra...@kde.org ---
commit 9c0dc6b3f0826d32eac310b2e7ecd858ca3df681
Author: Dan Vrátil dvra...@redhat.com
Date:   Mon Jun 29 22:45:11 2015 +0200

Don't leak old external payload files

Actually delete old payload files after we increase the payload revision or
switch from external to internal payload. This caused
~/.local/share/akonadi/file_db_data
to grow insanely for all users, leaving them with many duplicated files
(just with
different revisions).

It is recommended that users run akonadictl fsck to clean up the leaked
payload
files.

Note that there won't be any more releases of Akonadi 1.13 (and this has
been
fixed in master already), so I strongly recommend distributions to pick
this
patch into their packaging.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #7 from Martin Steigerwald m...@teamix.de ---
Bernhard, did you try anything to mitigate / work-around the issue? I suggest
at least a run of akonadictl fsck as I show it below. But you may also try
the SizeTreshold=32768 thing to mitigate the duplicates issues at least a bit –
maybe, I am not 100% positive that it help, but I think it does, cause it
causes payloads smaller than 32 KiB to go into the database and thus will in
general reduce amount of files in file_db_data. By this I think it will reduce
the chance of duplicates being created by a confused Akonadi server.

I thought I added it to the bug report, but I think there are at least two open
regarding file_db_data.

Ah, I mentioned in the other bug:

https://bugs.kde.org/show_bug.cgi?id=338402#c9
https://bugs.kde.org/show_bug.cgi?id=338402#c11

(These are really two different issues, one is about duplicates, one is about
the hugeness of that cache and probably also left over files or just too long
cache hold duration, since akonadictl fsck often moves lots of the files to
file_db-lost+found)


On my work account a huge Exchange IMAP:

I still do have duplicates, but not nearly as much after setting
SizeTreshold=32768 in [%General] of .config/akonadiserverrc

ms@merkaba:~/.local/share/akonadi/file_db_data fdupes -m .
3753 duplicate files (in 1010 sets), occupying 2495.1 megabytes

Wow, but still, that is 2,4 GiB of larger files then.

Total figures:

ms@merkaba:~/.local/share/akonadi du -sh *db_data
9,1Gdb_data
2,9Gfile_db_data



On my private account (POP3 and a smaller IMAP) things seem sane:

martin@merkaba:~/.local/share/akonadi du -sh *db_data
2,7Gdb_data
3,5Mfile_db_data

ms@merkaba:~/.local/share/akonadi/file_lost+found fdupes -m .
2916 duplicate files (in 686 sets), occupying 2206.8 megabytes

Now

ms@merkaba:~/.local/share/akonadi rm -r file_lost+found 
ms@merkaba:~/.local/share/akonadi

and more free space. :)

Daniel told several times that it is safe to remove file_lost+found.


Lets fsck the work one. Ok, there we go:

ms@merkaba:~/.local/share/akonadi/file_db_data fdupes -m .
60 duplicate files (in 52 sets), occupying 8.9 megabytes

.



See also:
https://lists.debian.org/debian-kde/2015/01/msg00055.html

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #9 from Martin Steigerwald mar...@lichtvoll.de ---
I set to confirmed and major importance as wasting 13 GiB IMHO is.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

Martin Steigerwald mar...@lichtvoll.de changed:

   What|Removed |Added

   Severity|normal  |major

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #8 from Martin Steigerwald m...@teamix.de ---
The akonadictl fsck should go before the size analysis and the rm -r
file_lost+found, sorry.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

Martin Steigerwald mar...@lichtvoll.de changed:

   What|Removed |Added

 Status|UNCONFIRMED |CONFIRMED
 Ever confirmed|0   |1

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Bernhard Jungk]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #10 from Bernhard Jungk f...@projectstarfire.de ---
Thanks for the advice, indeed it seems to be better now. I'll keep an eye on
that folder for a while.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Bernhard Jungk]

https://bugs.kde.org/show_bug.cgi?id=341884

Bernhard Jungk f...@projectstarfire.de changed:

   What|Removed |Added

 CC||f...@projectstarfire.de

--- Comment #6 from Bernhard Jungk f...@projectstarfire.de ---
This problem is becoming unbearable for me. :-(

~/.local/share/akonadi/file_db_data$ fdupes -m .
84860 duplicate files (in 3857 sets), occupying 13738.0 megabytes

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Felix Yan]

https://bugs.kde.org/show_bug.cgi?id=341884

Felix Yan felixonm...@archlinux.org changed:

   What|Removed |Added

 CC||felixonm...@archlinux.org

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #5 from Martin Steigerwald mar...@lichtvoll.de ---
Wow! Okay, I think thats a clear bug.

1) There are duplicates at all.

2) Akonadi does not clean up after itself.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

Martin Steigerwald mar...@lichtvoll.de changed:

   What|Removed |Added

 CC||mar...@lichtvoll.de

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[m . eik michalke]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #4 from m.eik michalke m...@reaktanz.de ---
thanks for the notice, i already cleared the lost+found folder right away then.

here's my duplicate statistics:

m@wurst ...ocal/share/akonadi/file_db_data $ fdupes -m .
6018 duplicate files (in 3955 sets), occupying 685.6 megabytes

cleaning up:

m@wurst ...ocal/share/akonadi/file_db_data $ akonadictl fsck
m@wurst ...ocal/share/akonadi/file_db_data $ fdupes -m .
307 duplicate files (in 291 sets), occupying 5.8 megabytes

so this is about one CD-ROM worth of junk in one month for me (since i last
cleaned up a month ago and removed the 14.5GB of duplicates which had
accumulated). the akonadictl fsck went really quick this time, the resulting
new lost+found folder contained 762MB in 9155 files.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

Martin Steigerwald m...@teamix.de changed:

   What|Removed |Added

 CC||m...@teamix.de

--- Comment #2 from Martin Steigerwald m...@teamix.de ---
Dan wrote several times that you can safely removed things in file_lost+found.
If you are unsure, maybe wait for a confirmation from him. On a Ext3/4
filesystem I would check its lost+found folder before removing things in there,
but it seems to Akonadi it may be different.

I think Akonadi shouldn´t be in need of a lost+found folder, as it is supposed
to be just a cache, but maybe thats something for Akonadi Next.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #3 from Martin Steigerwald m...@teamix.de ---
Okay, I do have some duplicates, but not all that much it seems, but I just did
a akonadictl fsck half an hour ago:

ms@merkaba:~/.local/share/akonadi/file_db_data fdupes -m .
1736 duplicate files (in 1729 sets), occupying 8.6 megabytes

Can you install fdupes (same package name on Debian) and check on your
file_db_data?

Thanks.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs

[Akonadi] [Bug 341884] dozens of duplicate mails in ~/.local/share/akonadi/file_db_data

[m . eik michalke]

https://bugs.kde.org/show_bug.cgi?id=341884

--- Comment #1 from m.eik michalke m...@reaktanz.de ---
i manually called akonadictl fsck. after waiting for some hours, it moved
over 110k of those duplicated mails to a new folder called file_lost+found.
the total disk usage for this unsused stuff is 14,5GB. i call that room for
improvement ;-)

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
Kdepim-bugs mailing list
Kdepim-bugs@kde.org
https://mail.kde.org/mailman/listinfo/kdepim-bugs