Re: KDEPIM 17.08: The whole thing works
Hi, On středa 22. listopadu 2017 16:31:21 CET Sandro Knauß wrote: > Hey, > > > I find using aptitude easier for upgrades. > > I just choose some major package like kdepim or kontact or kmail for upgrade > > and then jump over broken packages with "b" until everything is ok, then > > press "g" for list of upgraded and not upgaded packages and skim thru not > > upgraded for any forgoten packages. > > that sounds like a lot of clicky-clicky :) > apt install -t experimental kontact finds a solution that works. Without > manually to solve broken packages. > Okay finding packages that could/should be updated to is not that easy. > Yeah, but it's my morning ritual ;) (and i'm little old-fashioned and like to choose upgrades by hand) And i don't upgrade everything to experimental, for example i was waiting for long time for Mesa and libva2. > > Btw. is there a plan for newer versions of kdepim? > > as always for Debian - it is ready when it is ready - as you see in other mail > thread - pino already started to package 17.08.3 and uploading to unstable. > Sorry i missed that, thanks (i'm not trying to push anyone to making new packages) > > I have problem with akonadi-ews , unable to send it's emails, which should > > already be fixed in 17.08.0, but i'm still unable to send emails (I noticed > > it after 4 days, with over 20 work emails in my outgoing folder ...) > > https://github.com/KrissN/akonadi-ews/issues/35 > > github is not the official channel to communicate with kde software. You > should use bugs.kde.org for this. Just a small subset of KDE developers care > about github... It's not my bug report ;) And author of akonadi-ews says here, that everything is working for him with newest kdepim (that's reason of my question on newer kdepim packages) - i have reverted to using SMTP for now. > > Best Regards, > > hefee > Thanks, Libor
Re: KDEPIM 17.08: The whole thing works
Hey, > I find using aptitude easier for upgrades. > I just choose some major package like kdepim or kontact or kmail for upgrade > and then jump over broken packages with "b" until everything is ok, then > press "g" for list of upgraded and not upgaded packages and skim thru not > upgraded for any forgoten packages. that sounds like a lot of clicky-clicky :) apt install -t experimental kontact finds a solution that works. Without manually to solve broken packages. Okay finding packages that could/should be updated to is not that easy. > Btw. is there a plan for newer versions of kdepim? as always for Debian - it is ready when it is ready - as you see in other mail thread - pino already started to package 17.08.3 and uploading to unstable. > I have problem with akonadi-ews , unable to send it's emails, which should > already be fixed in 17.08.0, but i'm still unable to send emails (I noticed > it after 4 days, with over 20 work emails in my outgoing folder ...) > https://github.com/KrissN/akonadi-ews/issues/35 github is not the official channel to communicate with kde software. You should use bugs.kde.org for this. Just a small subset of KDE developers care about github... Best Regards, hefee signature.asc Description: This is a digitally signed message part.
Re: KDEPIM 17.08: The whole thing works
Hi, it works,hurray ;) I find using aptitude easier for upgrades. I just choose some major package like kdepim or kontact or kmail for upgrade and then jump over broken packages with "b" until everything is ok, then press "g" for list of upgraded and not upgaded packages and skim thru not upgraded for any forgoten packages. After upgrade akonadi started it's db upgrade, but my FS became full, i noticed it after day of my disk LED constantly blinking. Maybe some warning/news in akonadi package should be in place? (about resource heavy upgrade of DB) So big thanks to maintainers for fresh packages. Btw. is there a plan for newer versions of kdepim? I have problem with akonadi-ews , unable to send it's emails, which should already be fixed in 17.08.0, but i'm still unable to send emails (I noticed it after 4 days, with over 20 work emails in my outgoing folder ...) https://github.com/KrissN/akonadi-ews/issues/35 Libor On pátek 17. listopadu 2017 0:01:45 CET Martin Steigerwald wrote: > Hello. > > Okay, so I installed Akonadi + KDEPIM 17.08 and the whole things works. > > I had some issues tough. > > What did I do? > > 1. apt install -t experimental kmail – kdewebdev got removed, it appears > it needs an update. Also kdepim related kio plugins got removed, maybe > thats intentional. > > 2. Rebooted and started KMail. Akonadi produced lots of database load > and after a while KMail complained it would not start. Akonadi was indeed > not running only > > 3. I then looked for old 16.04 packages and did the following: > > apt install -t experimental libkf5akonadisearch-data libkf5akonadisearchxapian5 libkf5akonadinotes5 libkf5akonadisearchcore5 > > apt install -t experimental libkf5libkdepim-plugins:amd64 > > apt purge kf5-kdepimlibs-kio-plugins kdepimlibs-data > > (old transitional packages) > > 4. I logged out and in again, making sure that all processes of the user are gone. > > 5. Again Akonadi stopped after a while, complaining it could not find MySQL > socket file, I did not look much closer, MySQL was running initially just > fine and there has been quite some MySQL load. > > 6. But this time KMail offered to start Akonadi again, which I did > > 7. Then KMail said that Akonadi is updating the database to improve > performance. > > 8. It took about 15 minutes or so. > > 9. KAlarm asked about upgrading archived alarms to a new format. > > > I was able to read news feeds in Akregator during the upgrade. Akgregator > is still not Akonadi based. But well whatever the reported crash on start > bug is about, I did not see it. > > > Thats for now. Time to sleep. > > Now let´s see whether it sends this mail :) > > Thanks, > [1] mailto:libor.kle...@bcom.cz [2] tel:+420377457676 [3] http://www.bcom.cz
Re: Akonadi database (war: Re: KDEPIM 17.08: The whole thing works)
On Freitag, 17. November 2017 16:11:39 CET Martin Steigerwald wrote: > Martin Steigerwald - 17.11.17, 11:27: > > However all of that is nothing that makes sense to discuss here – as that > > are upstream matters. You may try to discuss this with upstream > > developers, > > but be prepared that they may not like the database or not discussion once > > again. This is really a question that I have seen *a ton of times* in > > various places. This is a touchy subject with upstream developers I think. > > > > I kindly ask you to drop the discussion on this matter here, as it would > > just be a waste of time here on this list. Thank you. > > > > Even if you go on… I may not reply anymore – I took a lot of time for this > > detailed reply already. See kdepim-users and kde-pim upstream mailing > > lists > > as well as various upstream bug reports or even discussions in debian-kde > > for tons of discussions on this matter. Anything is there, including the > > hints about optimizing the database. > > > > Yes, I am a bit annoyed that you brought the database or not question up > > once again. It has been discussed to death already. However, I also > > understand that there is no central place for all of the information > > […] > > I am sorry, Frank. > > This was partly uncalled for as I understand after doing some releasing. > > I still think these are upstream matters, however it was perfectly within my > responsibility if I answer and if so how much time I take for it. Did you actually see my other mail? Again, my question was not about having a DB or not, it was about whether it's possible to take a shortcut during KDE upgrade or not and about possible consequences. And it was definitely not meant to start another discussion. I'm still interested in the answer to my question. Kind regards, Frank
Re: KDEPIM 17.08: The whole thing works
Hello Sandro. Martin Steigerwald - 17.11.17, 14:04: > > 3. akonadictl start > > > > But I had issues with the database upgrade. Because it needs to create a > > temporary table, that was too big for my tmp filesystem > > The error is: > > Got error 64 'Temp file write failure' from InnoDB […] > > I solved this with setting the tempdir in > > ~/.local/share/akonadi/mysql.conf > > [mysqld] > > tmpdir = /usefullpath > > Maybe it would be good to implement to safeguard this in upstream code base > somehow. I.e. check free space beforehand and notify the user if there is > not enough. […] > IMHO thats another disadvantage of using MySQL/MariaDB by default. Much > temporary space needed for… things like this. First I told Frank to drop the topic, and then, as I sense a Debian KDE user who also works as Debian Qt/KDE team packager and as an upstream developer, I bring it up myself. Sandro, feel free to just ignore my nitpicking. Thank you, -- Martin
Re: Akonadi database (war: Re: KDEPIM 17.08: The whole thing works)
Hello Frank. Martin Steigerwald - 17.11.17, 11:27: > However all of that is nothing that makes sense to discuss here – as that > are upstream matters. You may try to discuss this with upstream developers, > but be prepared that they may not like the database or not discussion once > again. This is really a question that I have seen *a ton of times* in > various places. This is a touchy subject with upstream developers I think. > > I kindly ask you to drop the discussion on this matter here, as it would > just be a waste of time here on this list. Thank you. > > Even if you go on… I may not reply anymore – I took a lot of time for this > detailed reply already. See kdepim-users and kde-pim upstream mailing lists > as well as various upstream bug reports or even discussions in debian-kde > for tons of discussions on this matter. Anything is there, including the > hints about optimizing the database. > > Yes, I am a bit annoyed that you brought the database or not question up > once again. It has been discussed to death already. However, I also > understand that there is no central place for all of the information […] I am sorry, Frank. This was partly uncalled for as I understand after doing some releasing. I still think these are upstream matters, however it was perfectly within my responsibility if I answer and if so how much time I take for it. Thanks, -- Martin
Re: KDEPIM 17.08: The whole thing works
Hey Sandro, Sandro Knauß - 17.11.17, 13:42: > > Okay, so I installed Akonadi + KDEPIM 17.08 and the whole things works. […] > 3. akonadictl start > > But I had issues with the database upgrade. Because it needs to create a > temporary table, that was too big for my tmp filesystem > The error is: > Got error 64 'Temp file write failure' from InnoDB Oh. I think as the largest table here, the part table is about 3,2 GiB, this still fitted into: martin@merkaba:~> LANG=C df -hT /tmp Filesystem Type Size Used Avail Use% Mounted on tmpfs tmpfs 4.7G 356K 4.7G 1% /tmp > I solved this with setting the tempdir in ~/.local/share/akonadi/mysql.conf > [mysqld] > tmpdir = /usefullpath Maybe it would be good to implement to safeguard this in upstream code base somehow. I.e. check free space beforehand and notify the user if there is not enough. I think using /tmp may be less than optimal on machines with little RAM. Sure if the machine has lots of RAM, like this ThinkPad T520 with 16 GiB of it, it is beneficial. But a machine with just 4 GiB may only have 1 GiB of /tmp in most desktop distributions that use tmpfs for /tmp. Maybe it would be better to use /var/tmp, however as that is on / filesystem for many setups… well… IMHO thats another disadvantage of using MySQL/MariaDB by default. Much temporary space needed for… things like this. > At least for me the switching folder is much faster (because now the > threading is cached and not recreated every time). Wow, yes! I did not notice that yet. This is really noticeable. Thanks for pointing it out. Other than that tuning MariaDB database did the trick, so mail filtering appears similary fast than before. I noticed something else: The expandable progress bar stuff in the bottom of the kmail window is gone. Well often it was not too useful anyway. Thanks, -- Martin
Re: KDEPIM 17.08: The whole thing works
Hey, > Okay, so I installed Akonadi + KDEPIM 17.08 and the whole things works. 1. akonadictl stop 2. I updated to experimental packages too. I used: apt install -t experimental kontact 2.1 I also had to update some packages by hand: apt install -t experimental libkf5akonadisearch-data libkf5akonadisearchxapian5 libkf5akonadinotes5 libkf5akonadisearchcore5 apt install -t experimental libkf5libkdepim-plugins:amd64 (the purging was already done by updating kontact) 3. akonadictl start But I had issues with the database upgrade. Because it needs to create a temporary table, that was too big for my tmp filesystem The error is: Got error 64 'Temp file write failure' from InnoDB I solved this with setting the tempdir in ~/.local/share/akonadi/mysql.conf [mysqld] tmpdir = /usefullpath After that the upgrade started successfully and after some minutes I had a fully working kontact. At least for me the switching folder is much faster (because now the threading is cached and not recreated every time). Best regards, sandro signature.asc Description: This is a digitally signed message part.
Re: KDEPIM 17.08: The whole thing works
Martin Steigerwald - 17.11.17, 11:01: > Performance-wise I am not very impressed so far. For me it appears mail > filtering probably even got a bit slower. However, its too soon so say > something for sure. I bet that has been due to the fact that Akonadi upgrade undid my MariaDB optimization in mysql.conf. I redid these optimizations at it feels to me that I am back at regular speed again. However, its a bit early to say this and its all my subjective impression anyway. Time will tell. -- Martin
Akonadi database (war: Re: KDEPIM 17.08: The whole thing works)
Hello Frank, Frank Mehnert - 17.11.17, 09:17: > Martin, > > thank you for sharing your experiences! > > Regarding the database optimization: Would it be possible to remove > the complete database in certain cases? AFAIR, the Akonadi DB does > not store the messages itself, it only stores indexing information, > probably also flags / message tags etc... Anything is possible. Will you do it? The current KDEPIM developers will not do it. And I am quite certain I can say this here from what I remember of past discussions on this matter. They replied to this question countless of times. Akonadi caches messages, but it does not store them permanently. This has been addressed several times in various places as well already. As far as I understand the changes in indexing that Daniel works on will mean that for indexing the mails, contacts, events, whatever do not need to go into the Akonadi database temporarily anymore. I totally get the skepticism about the database, but a database does not need to be an issue. Zimbra uses a MySQL database too and its blazingly fast even with 45+ mails in *one* folder. However what I dislike a bit is that Akonadi uses a database, yet does not really totally free the user from handling it. Anyone with a larger set of mails needs to tune the MySQL database as I described here and on kdepim-users before… wait… maybe that is the performance drop I currently see in Akonadi/KDEPIM 17.08 Yep, the upgrade undid my optimizations: -innodb_buffer_pool_size=1024M +innodb_buffer_pool_size=128M (this is already more than the 64 MiB it used initially) # Buffer size used to write to the log files on disk (default:1M for builtin, 8M for plugin) # larger values means less I/O -innodb_log_buffer_size=32M +innodb_log_buffer_size=1M (and yes, I think I still have an bug report this open in upstream bugtracker) I suggest anyone with a large body of mails and stuff to run mysqltuner.pl on the database and tune it. I will redo the tuning now against the newly installed database configuration. I think this will fix the performance drop in mail filtering I perceived. So all that is my main critique regarding the database: The user should not be required to do things like this. So either Akonadi could handle MySQL or PostgreSQL with high performance *all by itself*, so maybe, just maybe, another database which does not require the user to be a part-time database admin would be in order. Another critique of mine is: Akonadi 16.04 at least did way to much and was still not really optimized as much as it can regarding database queries. KDEPIM developers are application developers, not really database experts, it appears to me. Additionally Akonadi 16.04 did to much on mail filtering with local maildirs anyway: It synchronized each folder after each few mails it placed there. Yet, no one else is changing those maildirs, it moved the mails, so why check in the first place? Synchronizing means it lists the directory contents and synchronizes it with the information it has in the database. Also with IMAP in 16.04 the folder synchronisation happened much too often. Even just after deleting a few mails. Actually I think a kind of database is needed. Going back to pre-akonadi times of using index files within KMail itself does not really make sense to me. The probable Akonadi replacement Sink that Kube mail client uses, also uses a database. I am not completely sure, but I think its LMDB, the Lightning Memory Database of the OpenLDAP project. It may be something else tough. But I would not hold my breath on Sink replacing Akonadi for KDEPIM anytime soon. Its the implementation that at least with 16.04 left a lot to be desired, not the usage of the database itself that is the main issue here. However all of that is nothing that makes sense to discuss here – as that are upstream matters. You may try to discuss this with upstream developers, but be prepared that they may not like the database or not discussion once again. This is really a question that I have seen *a ton of times* in various places. This is a touchy subject with upstream developers I think. I kindly ask you to drop the discussion on this matter here, as it would just be a waste of time here on this list. Thank you. Even if you go on… I may not reply anymore – I took a lot of time for this detailed reply already. See kdepim-users and kde-pim upstream mailing lists as well as various upstream bug reports or even discussions in debian-kde for tons of discussions on this matter. Anything is there, including the hints about optimizing the database. Yes, I am a bit annoyed that you brought the database or not question up once again. It has been discussed to death already. However, I also understand that there is no central place for all of the information I provided or hinted at above. I think at least the database optimization should go into upstream wiki, but I never took the
Re: KDEPIM 17.08: The whole thing works
Martin Steigerwald - 17.11.17, 00:01: > Okay, so I installed Akonadi + KDEPIM 17.08 and the whole things works. Some additional notes: It triggers a complete re-indexing of all mails. I aborted it now, cause I have *tons* of mails in folders for archival and I am happy for now when its just indexes with what I work with for now. Also Daniel Vrátil works on considerable changes to the indexing architecture that will make a complete rebuild of the index necessary anyway: Randa Report Part 2 https://www.dvratil.cz/2017/09/randa-report-part-2/ Performance-wise I am not very impressed so far. For me it appears mail filtering probably even got a bit slower. However, its too soon so say something for sure. I did not see many visible changes in KMail so far. Mail source view syntax highlighted now :) I reviewed KDE Application changelogs. Some things that stood out to me. In each release there is a huge ton of bug fixes according to full changelogs. I mainly looked about Akonadi + KDEPIM Runtime, KMail and Akregator, as that is what I mainly use. https://www.kde.org/announcements/ https://www.kde.org/announcements/announce-applications-17.08.0.php - KMail can use external editor, implemented by plugin - Bugfixes and regex line editor in Sieve editor - Some changes in KMail transport that I do not really understand - Seamonkey import in Akonadi import wizard - Updates to KMail documentation https://www.kde.org/announcements/announce-applications-17.04.0.php - Akonadi import wizard can import Sieve rules (I am not sure whether this wizard is something introduced with 16.12… or was just marked as new there due to packaging splits, which brought us the delay in NEW queue) - Better performance for search dialog by caching collection paths https://www.kde.org/announcements/announce-applications-16.12.0.php: - KMail and Akregator can use Google Safe Browsing to check if a link being clicked is malicious. Both have also added back printing support (needs Qt 5.8). => well I will disable that one, should I find in settings, okay, in KMail it is disabled by default (under "Security / Reading"), as well as in Akregator (under "Browser") https://www.kde.org/announcements/announce-applications-16.08.0.php: - more Qt WebEngine (instead of Webkit, I think KMail) - detailed changelog: quicker access to search (via class instead of DBUS) This is just what I found. I bet I may have overlooked quite a bit and there can be stuff that is important to you, but not to me. Thanks, -- Martin
Re: KDEPIM 17.08: The whole thing works
Martin, thank you for sharing your experiences! Regarding the database optimization: Would it be possible to remove the complete database in certain cases? AFAIR, the Akonadi DB does not store the messages itself, it only stores indexing information, probably also flags / message tags etc... Frank
KDEPIM 17.08: The whole thing works
Hello. Okay, so I installed Akonadi + KDEPIM 17.08 and the whole things works. I had some issues tough. What did I do? 1. apt install -t experimental kmail – kdewebdev got removed, it appears it needs an update. Also kdepim related kio plugins got removed, maybe thats intentional. 2. Rebooted and started KMail. Akonadi produced lots of database load and after a while KMail complained it would not start. Akonadi was indeed not running only 3. I then looked for old 16.04 packages and did the following: apt install -t experimental libkf5akonadisearch-data libkf5akonadisearchxapian5 libkf5akonadinotes5 libkf5akonadisearchcore5 apt install -t experimental libkf5libkdepim-plugins:amd64 apt purge kf5-kdepimlibs-kio-plugins kdepimlibs-data (old transitional packages) 4. I logged out and in again, making sure that all processes of the user are gone. 5. Again Akonadi stopped after a while, complaining it could not find MySQL socket file, I did not look much closer, MySQL was running initially just fine and there has been quite some MySQL load. 6. But this time KMail offered to start Akonadi again, which I did 7. Then KMail said that Akonadi is updating the database to improve performance. 8. It took about 15 minutes or so. 9. KAlarm asked about upgrading archived alarms to a new format. I was able to read news feeds in Akregator during the upgrade. Akgregator is still not Akonadi based. But well whatever the reported crash on start bug is about, I did not see it. Thats for now. Time to sleep. Now let´s see whether it sends this mail :) Thanks, -- Martin
