Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 23/04/2014, at 7:32 AM, Burkhard Lück wrote: Am Mittwoch, 23. April 2014, 06:55:23 schrieb Ian Wadham: On 20/04/2014, at 6:12 PM, Burkhard Lück wrote: Am Sonntag, 20. April 2014, 16:41:35 schrieb Ian Wadham: Well, that is news to me. I never knew before that all the KDE Handbooks are on the web now. What about translations? http://docs.kde.org/development/de/kdegames/palapeli/large-puzzle-holders. html Thanks, Burkhardt. I hope you did not find it too tedious translating that stuff … :-) No It was a pleasure and while translating it, I realized how many ideas/thoughts/effort you had invested for the difficulties to solve a big puzzle on a relative small screen. Thank you again, Burkhardt. That was a very nice compliment. Cheers, Ian W. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 20/04/2014, at 6:12 PM, Burkhard Lück wrote: Am Sonntag, 20. April 2014, 16:41:35 schrieb Ian Wadham: Well, that is news to me. I never knew before that all the KDE Handbooks are on the web now. What about translations? http://docs.kde.org/development/de/kdegames/palapeli/large-puzzle-holders.html Thanks, Burkhardt. I hope you did not find it too tedious translating that stuff … :-) Cheers, Ian W. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Am Mittwoch, 23. April 2014, 06:55:23 schrieb Ian Wadham: On 20/04/2014, at 6:12 PM, Burkhard Lück wrote: Am Sonntag, 20. April 2014, 16:41:35 schrieb Ian Wadham: Well, that is news to me. I never knew before that all the KDE Handbooks are on the web now. What about translations? http://docs.kde.org/development/de/kdegames/palapeli/large-puzzle-holders. html Thanks, Burkhardt. I hope you did not find it too tedious translating that stuff … :-) No It was a pleasure and while translating it, I realized how many ideas/thoughts/effort you had invested for the difficulties to solve a big puzzle on a relative small screen. Thanks a lot. -- Burkhard Lück Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Am Sonntag, 20. April 2014, 16:41:35 schrieb Ian Wadham: Well, that is news to me. I never knew before that all the KDE Handbooks are on the web now. What about translations? http://docs.kde.org/development/de/kdegames/palapeli/large-puzzle-holders.html -- Burkhard Lück Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 18/04/2014, at 4:32 AM, Luigi Toscano wrote: Ian Wadham ha scritto: Hello Thomas and Luigi, Sorry it has been such a while since you wrote. A lot of water has flowed under the bridge since then, but this issue is still of the utmost importance to MacPorts. See: https://trac.macports.org/wiki/KDEProblems/KDETickets On 21/03/2014, at 11:52 PM, Thomas Lübking wrote: On Freitag, 21. März 2014 08:24:06 CEST, Ian Wadham wrote: That call to KGlobal::locale(); seems an odd one, KDE guys. That function is supposed to return a locale (KLocale *), but here it is executed as a procedure, ignoring the return result. I can only conclude that the code is being executed for its side-effects: It will likely be to call protected KLocale::initInstance(), eventually to intantiate it from the main thread for sure. Not sure if it's required at all - look at the date of the commit! commit 693da1d1df4876d7c898f3035beead76288872d5 Author: Stephan Kulow ..@kde.org Date: Fri Jul 6 15:19:46 2001 + update to docbook-xsl 1.40 [] -KGlobal::locale()-setMainCatalogue(kio_help); +KLocale::setMainCatalogue(kio_help); KInstance ins(meinproc); +KGlobal::locale(); [….] I hesitate to touch even one line of code from Stephan, one of the all-time greats of KDE, but I think that is what I am going to have to do. I wrote to Albert Astals Cid, to find out if meinproc4_simple could do the job of producing KDE Handbooks on MacPorts and Apple OS X, but it can not: it does not include compression. The best solution IMHO will be something like meinproc4_simple. It would bypass all the legacy code and just process command-lines of the form: meinproc4 [--check] [--cache] HandbookPath/index.cache.bz2 DocbookPath/index.docbook which is what CMake's kde4_create_handbook() macro boils down to in the end. The alternatives would be to re-work the mainline code of meinproc4 OR to try and investigate these intermittent crashes across the four versions of Apple OS X and three Mac hardware architectures supported by MacPorts. The crashes affect some people's Apple machines but not others --- and not mine, for example. Thanks for looking into it, just to question here: - did you try to just disable that line? It's certainly less breaking than try to rewrite a tool where locale support have been rewritten in KF5. On the other side, if the call to that Mac system functions produces some issues, it could potentially affect even newer applications (I haven't checked how Qt5 implements locale support). No point. meinproc4 *never* fails on my configuration, nor on Nicos' system (he is the current maintainer of all KDE ports on MacPorts). Also, I am by no means certain that the 'KGlobal::locale();' line is the only point of failure. It is just where it failed in the only case where we have a backtrace. - would it be possible to create such a map (combination of arch/SO where it crashes)? That would require a lot of co-operation by a whole lot of MacPorts people who are not at all happy about the meinproc4 crashes and the lack of response to https://bugs.kde.org/show_bug.cgi?id=261509 over more than 3 years. Two key MacPorts/KDE maintainers did quit over this and other problems. Marko, Mario and I are just beginning to patch things up again on the kde-mac list. I was hoping, rather, to present the MacPorts developers with a crash-proofed version of meinproc4 for them to test. It just need a machine where the crash can be reproduced with a debug build and the debugger. The impossible I do today: miracles take a little longer. … :-) Cheers, Ian W. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 18/04/2014, at 4:58 AM, Thomas Lübking wrote:
On Donnerstag, 17. April 2014 20:32:17 CEST, Luigi Toscano wrote:
Ian Wadham ha scritto:
Sorry it has been such a while since you wrote. A lot of water has
flowed under the bridge since then, but this issue is still of the utmost
importance to MacPorts. See:
https://trac.macports.org/wiki/KDEProblems/KDETickets ...
Thanks for looking into it, just to question here:
- did you try to just disable that line? It's certainly less breaking than
try to rewrite a tool where locale support have been rewritten in KF5.
If removing the KLocale() constructor avoids it, i'm fairly sure it will be
the bogus CFStringGetLength call, so to me it would seem more reasonable to
protect convert_CFString_to_QString
kdelibs/kdecore/kernel/kkernel_mac.cpp
---
QString convert_CFString_to_QString(CFStringRef str) {
+if (str == NULL) {
+return QString();
+}
eventually print a warning (while i've no idea what this condition implies,
like eg. a broken setup. It could be a bug in CFStringRef or CFLocaleGetValue
or either isn't re-entrant or whatever)
And no, forking the application seems the worst option
Okay, so how about the following idea?
meinproc4 is essentially what, back in the day, we used to call a batch
process. It runs with no GUI and without interaction, often as part of a
larger batch process, such as a MacPorts build. It should not crash,
taking everything else with it, but should fail gracefully. The worst
that should happen is that one KDE Handbook does not get built.
This is much better than having *no* KDE Handbooks built, which has
been the default in MacPorts for a few years now, because of the risk
of meinproc4 crashes.
So, can we add a signal trap to meinproc4, so that it issues a heavy
warning message, but then exits normally, leaving the rest of the run
to continue? Or can the kde4_create_handbook() macro be modified,
so that if meinproc4 crashes we automatically produce debugging
info and just carry on?
There must be ways to do this within KDE or Qt or C++ even.
All the best, Ian W.
Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Hello Thomas and Luigi, Sorry it has been such a while since you wrote. A lot of water has flowed under the bridge since then, but this issue is still of the utmost importance to MacPorts. See: https://trac.macports.org/wiki/KDEProblems/KDETickets On 21/03/2014, at 11:52 PM, Thomas Lübking wrote: On Freitag, 21. März 2014 08:24:06 CEST, Ian Wadham wrote: That call to KGlobal::locale(); seems an odd one, KDE guys. That function is supposed to return a locale (KLocale *), but here it is executed as a procedure, ignoring the return result. I can only conclude that the code is being executed for its side-effects: It will likely be to call protected KLocale::initInstance(), eventually to intantiate it from the main thread for sure. Not sure if it's required at all - look at the date of the commit! commit 693da1d1df4876d7c898f3035beead76288872d5 Author: Stephan Kulow ..@kde.org Date: Fri Jul 6 15:19:46 2001 + update to docbook-xsl 1.40 [] -KGlobal::locale()-setMainCatalogue(kio_help); +KLocale::setMainCatalogue(kio_help); KInstance ins(meinproc); +KGlobal::locale(); [….] I hesitate to touch even one line of code from Stephan, one of the all-time greats of KDE, but I think that is what I am going to have to do. I wrote to Albert Astals Cid, to find out if meinproc4_simple could do the job of producing KDE Handbooks on MacPorts and Apple OS X, but it can not: it does not include compression. The best solution IMHO will be something like meinproc4_simple. It would bypass all the legacy code and just process command-lines of the form: meinproc4 [--check] [--cache] HandbookPath/index.cache.bz2 DocbookPath/index.docbook which is what CMake's kde4_create_handbook() macro boils down to in the end. The alternatives would be to re-work the mainline code of meinproc4 OR to try and investigate these intermittent crashes across the four versions of Apple OS X and three Mac hardware architectures supported by MacPorts. The crashes affect some people's Apple machines but not others --- and not mine, for example. The question remains of how to present the source-code. Rather than write a separate source file and build a separate executable, with a different name, as has been done with meinproc4_simple, I am thinking of using conditional code to produce an executable that is either the full meinproc4 or a stripped down meinproc4, for use in Apple OS X builds only. That way, all the CMake stuff can stay the same as it is now. The stripped-down meinproc4 would issue a warning message if it is given command-line syntax it cannot handle, but would NOT crash the whole build. I think the best way might be to start off the code with something like: #ifndef Q_OS_MAC #define MEINPROC4_FULL #endif then use #ifdef MEINPROC4_FULL or #ifndef MEINPROC4_FULL to control what gets complied on Apple OS X or elsewhere. The idea is that one could define MEINPROC4_FULL externally on Apple OS X and still get the full version of meinproc4 if it was needed. Please let me know what you think of these ideas. Please also let me know what group(s) or individual(s) to post to on ReviewBoard for this work. Cheers, Ian W. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Ian Wadham ha scritto: Hello Thomas and Luigi, Sorry it has been such a while since you wrote. A lot of water has flowed under the bridge since then, but this issue is still of the utmost importance to MacPorts. See: https://trac.macports.org/wiki/KDEProblems/KDETickets On 21/03/2014, at 11:52 PM, Thomas Lübking wrote: On Freitag, 21. März 2014 08:24:06 CEST, Ian Wadham wrote: That call to KGlobal::locale(); seems an odd one, KDE guys. That function is supposed to return a locale (KLocale *), but here it is executed as a procedure, ignoring the return result. I can only conclude that the code is being executed for its side-effects: It will likely be to call protected KLocale::initInstance(), eventually to intantiate it from the main thread for sure. Not sure if it's required at all - look at the date of the commit! commit 693da1d1df4876d7c898f3035beead76288872d5 Author: Stephan Kulow ..@kde.org Date: Fri Jul 6 15:19:46 2001 + update to docbook-xsl 1.40 [] -KGlobal::locale()-setMainCatalogue(kio_help); +KLocale::setMainCatalogue(kio_help); KInstance ins(meinproc); +KGlobal::locale(); [….] I hesitate to touch even one line of code from Stephan, one of the all-time greats of KDE, but I think that is what I am going to have to do. I wrote to Albert Astals Cid, to find out if meinproc4_simple could do the job of producing KDE Handbooks on MacPorts and Apple OS X, but it can not: it does not include compression. The best solution IMHO will be something like meinproc4_simple. It would bypass all the legacy code and just process command-lines of the form: meinproc4 [--check] [--cache] HandbookPath/index.cache.bz2 DocbookPath/index.docbook which is what CMake's kde4_create_handbook() macro boils down to in the end. The alternatives would be to re-work the mainline code of meinproc4 OR to try and investigate these intermittent crashes across the four versions of Apple OS X and three Mac hardware architectures supported by MacPorts. The crashes affect some people's Apple machines but not others --- and not mine, for example. Thanks for looking into it, just to question here: - did you try to just disable that line? It's certainly less breaking than try to rewrite a tool where locale support have been rewritten in KF5. On the other side, if the call to that Mac system functions produces some issues, it could potentially affect even newer applications (I haven't checked how Qt5 implements locale support). - would it be possible to create such a map (combination of arch/SO where it crashes)? It just need a machine where the crash can be reproduced with a debug build and the debugger. That way, all the CMake stuff can stay the same as it is now. The stripped-down meinproc4 would issue a warning message if it is given command-line syntax it cannot handle, but would NOT crash the whole build. I think the best way might be to start off the code with something like: #ifndef Q_OS_MAC #define MEINPROC4_FULL #endif Again, you just need to pass the language, if you disable that crashing line. No need to fork the application just for MacOSX. Ciao -- Luigi Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On Donnerstag, 17. April 2014 20:32:17 CEST, Luigi Toscano wrote:
Ian Wadham ha scritto:
Sorry it has been such a while since you wrote. A lot of water has
flowed under the bridge since then, but this issue is still of the utmost
importance to MacPorts. See:
https://trac.macports.org/wiki/KDEProblems/KDETickets ...
Thanks for looking into it, just to question here:
- did you try to just disable that line? It's certainly less breaking than
try to rewrite a tool where locale support have been rewritten
in KF5.
If removing the KLocale() constructor avoids it, i'm fairly sure it will be the
bogus CFStringGetLength call, so to me it would seem more reasonable to protect
convert_CFString_to_QString
kdelibs/kdecore/kernel/kkernel_mac.cpp
---
QString convert_CFString_to_QString(CFStringRef str) {
+if (str == NULL) {
+return QString();
+}
eventually print a warning (while i've no idea what this condition implies,
like eg. a broken setup. It could be a bug in CFStringRef or CFLocaleGetValue
or either isn't re-entrant or whatever)
And no, forking the application seems the worst option (remeber Ian, you'd have
to maintain that fork ;-)
Cheers,
Thomas
Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 20/03/2014, at 7:28 AM, mk-li...@email.de wrote: On 19 Mar 2014, at 06:29 , Thomas Lübking thomas.luebk...@gmail.com wrote: There seems a known issue reg. multithreaded libxml2 [1], but since Marko was the reporter, i simply ruled it out being the remaining one. I doubt it was a libxml2 issue, since the corresponding poster wrote — pHi on windows we had a similar crash./p divThe problem was in libxml2, when builded with multithread support.Disabling multithread fixed it./div — The crash I described back then happened with KMyMoney, but did occur for any other KDE software arbitrarily every now and then. I think in order to reproduce the error I’d also need to build stuff highly parallel with all 8 cores and in an an endless loop, but I’d need to get familiar with parallel and stuff alike, I am afraid… Well, let’s see what Ian can come up with once he’s done with partying his birthday. ;-) Heh! Well, I am not any kind of KDE genius and kdoctools is foreign territory for me. It's hard enough for me to write that .docbook format of documentation … :-) I tried a little script to get meinproc4 to fail by executing several copies in parallel. for game in killbots kjumpingcube konquest kpat kubrick kgoldrunner palapeli do echo meinproc4 for $game cd /kdedev/games/$game/doc /opt/local/bin/meinproc4 --check --cache /kdedev/build/games/$game/doc/index.cache.bz2 /kdedev/games/$game/doc/index.docbook done IOW, run meinproc4 for seven games Handbooks at once. The main command is what eventuates from CMake and make when you are installing documentation. /keddev is my KDE development area and /opt/local/bin is where MacPorts puts utilities (though not GUI applications, which require to be installed in a special way on Mac OS X). It was all over in a second or two, with a spike of about 1.5 cores on Apple's Activity Monitor. No crashes. But then meinproc4 never fails for me. Then I wrapped a do forever loop around the above --- and then it crashed, but it was hardly a fair test: writing seven output files an unknown number of times at once. So I think we can discount the concurrency-problem theory. In any case, I had a look at https://trac.macports.org/attachment/ticket/41326/main.log a huge log file (25Mb) from a crash of meinproc4. I do not know what this run was doing (there seems to be no replay of the MacPorts command that started it), but there seem to be multiple commands that are aimed at building meinproc4 itself. What really happened? Anyway, there is just one attempt to execute meinproc4, AFAICS, and that is right near the end of that huge log file. So no concurrency. Anyway, the one backtrace we have (not from this run BTW) shows meinproc4 failing on its one and only call to KGlobal::locale(); at line 109 of file meinproc.cpp. That is part of meinproc4's initialisation, before it starts processing any input. That call to KGlobal::locale(); seems an odd one, KDE guys. That function is supposed to return a locale (KLocale *), but here it is executed as a procedure, ignoring the return result. I can only conclude that the code is being executed for its side-effects, see: http://api.kde.org/4.x-api/kdelibs-apidocs/kdecore/html/kglobal_8cpp_source.html#l00144 Macports guys, in https://trac.macports.org/attachment/ticket/41326/main.log, meinproc4 is actually running inside a script called meinproc4.shell, which I think may be generated by MacPorts. I wonder if meinproc4 has not been fully installed yet (it seems to be being built earlier on in the run). If so, I wonder if it is running from somewhere that is not its usual install location (i.e. not /opt/local/bin/meinproc4) or maybe it has been deprived of its usual KDE environment setup in some way, so that is why it cannot find a locale. One final idea. I see that there is a version of meinproc4 called meinproc4_simple, which the KDE translators use for some purpose which I cannot quite make out. Anyway, it appears to be a special-purpose version of meinproc4 and it is essentially a Qt-only application. I wonder if we could legislate the MacPorts meinproc4 bug out of existence by making another special-purpose version of meinproc4 that only takes in a .docbook file, checks it and spits out a .cache.bz2 file, thus omitting the code where meinproc4 appears to be crashing in the Apple OS X environment. That version could then be the one used to create Handbooks across all platforms. Just a thought. Cheers, Ian W. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On Freitag, 21. März 2014 08:24:06 CEST, Ian Wadham wrote: That call to KGlobal::locale(); seems an odd one, KDE guys. That function is supposed to return a locale (KLocale *), but here it is executed as a procedure, ignoring the return result. I can only conclude that the code is being executed for its side-effects: It will likely be to call protected KLocale::initInstance(), eventually to intantiate it from the main thread for sure. Not sure if it's required at all - look at the date of the commit! commit 693da1d1df4876d7c898f3035beead76288872d5 Author: Stephan Kulow ..@kde.org Date: Fri Jul 6 15:19:46 2001 + update to docbook-xsl 1.40 [] -KGlobal::locale()-setMainCatalogue(kio_help); +KLocale::setMainCatalogue(kio_help); KInstance ins(meinproc); +KGlobal::locale(); [] Cheers, Tomas Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 19/03/2014, at 4:29 PM, Thomas Lübking wrote: YY H H YYY H H PP PP YYY YYY HHH HHH PPPPP YYY YYY HHH HHHA PP PP PPPPYYY YYY Thank you very much Thomas Lübking, Thomas (Baumgart) and Marko for the birthday greetings. The above came out looking great once I had it copied and pasted into a fixed font … :-) Cheers, Ian W. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 19 Mar 2014, at 06:29 , Thomas Lübking thomas.luebk...@gmail.com wrote: There seems a known issue reg. multithreaded libxml2 [1], but since Marko was the reporter, i simply ruled it out being the remaining one. I doubt it was a libxml2 issue, since the corresponding poster wrote — pHi on windows we had a similar crash./p divThe problem was in libxml2, when builded with multithread support.Disabling multithread fixed it./div — The crash I described back then happened with KMyMoney, but did occur for any other KDE software arbitrarily every now and then. I think in order to reproduce the error I’d also need to build stuff highly parallel with all 8 cores and in an an endless loop, but I’d need to get familiar with parallel and stuff alike, I am afraid… Well, let’s see what Ian can come up with once he’s done with partying his birthday. ;-) Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
mk-li...@email.de ha scritto: On 19 Mar 2014, at 06:29 , Thomas Lübking thomas.luebk...@gmail.com wrote: There seems a known issue reg. multithreaded libxml2 [1], but since Marko was the reporter, i simply ruled it out being the remaining one. I doubt it was a libxml2 issue, since the corresponding poster wrote — pHi on windows we had a similar crash./p divThe problem was in libxml2, when builded with multithread support.Disabling multithread fixed it./div — The crash I described back then happened with KMyMoney, but did occur for any other KDE software arbitrarily every now and then. The crash described in that stack trace happens in a part of code which is executed *before* initializing libxml. Here: http://lxr.kde.org/source/kde/kdelibs/kdoctools/meinproc.cpp#109 Ciao -- Luigi Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 19 Mar 2014, at 21:50 , Luigi Toscano luigi.tosc...@tiscali.it wrote: The crash described in that stack trace happens in a part of code which is executed *before* initializing livxml. OK, and what do we learn from that? Greets, Marko Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 19 Mar 2014, at 22:30 , Thomas Lübking thomas.luebk...@gmail.com wrote: That the libxml2 bug is not related to the bug #261509 backtrace (doesn't change anything since i anticipated that for social reasons ;-) Ah, ok, so that supports the notion that it was just an accidental coincidence. So, I am afraid we might really have what Ian suggested. A problem with concurrently running processes. meinproc uses some cache, does it perhaps cross-use data cached by other instances of meinproc and that collides with itself if there are parallel tasks running? Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On Mittwoch, 19. März 2014 22:24:13 CEST, mk-li...@email.de wrote: On 19 Mar 2014, at 21:50 , Luigi Toscano luigi.tosc...@tiscali.it wrote: The crash described in that stack trace happens in a part of code which is executed *before* initializing livxml. OK, and what do we learn from that? That the libxml2 bug is not related to the bug #261509 backtrace (doesn't change anything since i anticipated that for social reasons ;-) Cheers, Thomas Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 20/03/2014, at 8:34 AM, mk-li...@email.de wrote: On 19 Mar 2014, at 22:30 , Thomas Lübking thomas.luebk...@gmail.com wrote: That the libxml2 bug is not related to the bug #261509 backtrace (doesn't change anything since i anticipated that for social reasons ;-) Ah, ok, so that supports the notion that it was just an accidental coincidence. So, I am afraid we might really have what Ian suggested. A problem with concurrently running processes. meinproc uses some cache, does it perhaps cross-use data cached by other instances of meinproc and that collides with itself if there are parallel tasks running? I think the cache is a type of *output* of meinproc4 --- some HTML files archived into one file and compressed (.bz2). That is what KDE routinely installs. Then if a user comes along and wants to read the doco, the Help system uncompresses and unpacks it. I am going off the concurrency bug theory a bit. In the one crash backtrace we have, I think meinproc4 has crashed in its initialisation phase (at line 109 in file meinproc.cpp), before it ever starts reading and parsing a .docbook file. Cheers, Ian W. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On Tuesday 18 of March 2014 15:09:08 Ian Wadham wrote: Hi Luigi, On 18/03/2014, at 10:17 AM, Luigi Toscano wrote: I'm pretty sure meinproc4 does not depend on anything tex-related. It depends on libxml, libxslt, docbook-xml and docbook-xslt. Could you please try to investigate a bit more about the reasons for this? There is a long-standing bug-report re meinproc4 on Apple OS X and MacPorts: https://bugs.kde.org/show_bug.cgi?id=261509 Which is not on kdoctools subproduct, I will fix it. First reported 2010-12-29. So far no response from KDE apart from Jekyll Wu enquiring whether it is still a problem (2013-03-01). Latest post from MacPorts 2014-01-02. MacPorts have also raised four tickets re this on their own bug-reporting database. Links to those are in the comments on https://bugs.kde.org/show_bug.cgi?id=261509 So, it seems never worked. This is the commit which introduced the macLocaleValue; it seems that the output of CFLocaleGetValue is invalid and then the crash: http://quickgit.kde.org/?p=kdelibs.gita=commith=e0973a28cbe112b1740af927e8383ba677068f80 (the current code is a bit different). /me summons John Layt I wonder why it happens only here and not in other applications; the crash is in a innocent call to KGlobal. Do you know about similar crashes/stacktraces in other applications? Apparently it crashes randomly on one document or another during a large install that calls for KDE documentation. In amongst the MacPorts tickets is at least one backtrace. I would like to follow that up, but I cannot find the meinproc4 source code repository anywhere in KDE Projects and a search comes up empty: https://projects.kde.org/search?projects=1q=meinproc4 As you found out it's part of kdelibs/kdoctools. In Frameworks it is part of a separate KDocTools framework. And yes, I'm the maintainer now. I would really need some help from a real Mac programmer to find why that snippet of code is crashing. Let's see if I can extract that code in simple test program (any help or volunteer is appreciated), but I would need some Mac around to test it. Luigi, can you tell me where, in the source-code tree, meinproc4 is? Also, is there a maintainer for it? If so, who is he or she? Bug 261509 is still at UNCONFIRMED status after four years and the only people on the mailing list for it are four guys from MacPorts. Its auto assignment is to kdelibs-b...@kde.org Yes, it was not exactly maintained - and the component is generic so it didn't end up in the right bucket. For sure it needs to be fixed, the downstream solution of removing documentation is non-solution. Ciao -- Luigi Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Hi Luigi, On 18 Mar 2014, at 14:33 , Luigi Toscano luigi.tosc...@tiscali.it wrote: I wonder why it happens only here and not in other applications; the crash is in a innocent call to KGlobal. Do you know about similar crashes/stacktraces in other applications? I don’t know whether this stack trace is representative, but the only one to start from. As pointed out this meinproc4 error always occurred only sporadically and unpredictably... :-( I would really need some help from a real Mac programmer to find why that snippet of code is crashing. Let's see if I can extract that code in simple test program (any help or volunteer is appreciated), but I would need some Mac around to test it. Luigi, if you can put or suggest a minimal test program it would be helpful. One could use SVN access to your folder at KDE’s server, BitBucket via Mercurial or GitHub, whatever you like, to share the code. I am ready to support you if you need someone to build stuff on MacOSX. I am not a Mac developer though, but do have some experience building and testing apps on Linux and Mac. For sure it needs to be fixed, the downstream solution of removing documentation is non-solution. No, it is certainly not. When I installed KDevelop [1] I realised already that tons of documentation is missing in its man page browser, although it would be very helpful, like all the KDE stuff. ;-) Luckily the great KDevelop code reader show a lot of things already. Greets, Marko [1] http://mail.kde.org/pipermail/kdevelop/2014-March/018250.html Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Ages ago I was hunting a but in KDE’s about dialog and had set up a Mercurial repo for it on BitBucket [1]. One could use that location for the minor app which you’re suggesting. [1] https://bitbucket.org/mkae/kde-tests Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
As shown in [1] I’ve got a working native KDE tutorial application started. So, now one could add whatever code you think is needed to test the issue observed with meinproc4. [1] http://mail.kde.org/pipermail/kdevelop/2014-March/018258.html Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
mk-li...@email.de wrote: Hi Luigi, On 18 Mar 2014, at 14:33 , Luigi Toscano luigi.tosc...@tiscali.it wrote: I wonder why it happens only here and not in other applications; the crash is in a innocent call to KGlobal. Do you know about similar crashes/stacktraces in other applications? I don’t know whether this stack trace is representative, but the only one to start from. As pointed out this meinproc4 error always occurred only sporadically and unpredictably... :-( Ok: I've seen the other message, but then I would start from this and make sure that the stacktrace is the correct one. Could you (or any other Mac user/developer) please try to - recompile kdelibs with debug symbols - run meinproc4 on any document using the debugger, to get a complete backtrace I guess that MacOSX uses lldb instead of gdb nowadays: I found this http://lldb.llvm.org/tutorial.html I shouldn't be too difficult, just run meinproc4 through the debugger with the specified docbook (any docbook provided by KDE software) with lldb until it crashes, and from its shell execute: (lldb) thread backtrace all This is important to understand if the issue is really that one. Ciao -- Luigi Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
On 18 Mar 2014, at 23:15 , Luigi Toscano luigi.tosc...@tiscali.it wrote: Ok: I've seen the other message, but then I would start from this and make sure that the stacktrace is the correct one. Hmm… Could you (or any other Mac user/developer) please try to - recompile kdelibs with debug symbols Right now I am not able to do so (and I knew you’d ask for it…), because I haven’t set up a parallel install for debugging. That’s something I long planned to do since I migrated slowly to my new machine, but I haven’t achieved it YET. - run meinproc4 on any document using the debugger, to get a complete backtrace Ha, I tell you, that will be impossible, I am afraid, because those crashes come unpredictably... :-( I guess that MacOSX uses lldb instead of gdb nowadays: I found this http://lldb.llvm.org/tutorial.html I shouldn't be too difficult, just run meinproc4 through the debugger with the specified docbook (any docbook provided by KDE software) with lldb until it crashes, and from its shell execute: (lldb) thread backtrace all This is important to understand if the issue is really that one. Yep I know, but there is NO specific docbook, at least to my knowledge, which would always crash it. I wonder what Ian can add in here, since he is building docs all the time recently… I guess one would have to automate the procedure and let meinproc4 again and again build one or several files. Perhaps that’s a possibility. But I don’t know whether this could be automated with lldb… Needs some investigation, I guess. OK, I had hoped you had an idea to test this hypothesis with a minimal app, but I see we might have to do brute force on meinproc4 itself. Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Luigi, before I am off for tonight I verified that I can start meinproc4 with lldb. Works. All I’d need now would be big and complex files to work on. Without doing any building of any port I’d want to just steal some XML/XSL source files perhaps from KDELIBS and let meinproc4 work on them in an endless loop. Perhaps I’d be able to see it crash in the debugger so that we get a few more stack traces. (Sorry, for now w/o debug symbols, but it’s a start.) Could you please point me to how I should organise which files in a test directory tree in such a way that meinproc would happily do its Sisyphus job?! OK, I am off for now, looking forward to suggestions now to get this going. :) Greets, Marko Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Thomas Lübking wrote: On Dienstag, 18. März 2014 23:26:34 CEST, mk-li...@email.de wrote: On 18 Mar 2014, at 23:15 , Luigi Toscano luigi.tosc...@tiscali.it wrote: Ok: I've seen the other message, but then I would start from this and make sure that the stacktrace is the correct one. Hmm… According to the backtrace in the bug, the str parameter here http://api.kde.org/4.x-api/kdelibs-apidocs/kdecore/html/kkernel__mac_8cpp.html#abd7a47c0501f4e72cfdb901f4f414248 is NULL, thus try with an initial if (str == NULL) return QString(); Yes, but it's more a workaround. It would be interesting to know why that string (which should contain some value, maybe) is NULL. (if the trace is really that one) Ciao -- Luigi Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
So, I got all *.docbook files coming with kdelibs4 and copied them into a test directory and wrote a bash script which calls meinproc4 for every of those files. Of course there are now tons of warning because I obviously don’t have some needed files in certain directories which meinproc4 needs… A few html files get generated, but many won’t. So, just in case you can be bothered to give me a more hands-on hint which set of files I could let meinproc4 work on, that would be really cool. I guess it would be worthwhile to have inclusion of other references in there and all the stuff which would occur in a real package build/install. I hope I am not asking for too much, but it seems tedious for me to start to analyse some Makefile's or whatever now to try to mimic something close to a real documentation build. OK, now I am definitely off. Far too late already. :( Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
Re: meinproc4 on Mac, KLocale (was Re: What to test for 4.13?)
Hi Thomas, thanks for that clarification with code! :-) OK, now I see what you mean. I still wonder how and where you found this str == NULL” issue… I must have seen some other stack trace then. Which one are you referring to, actually? Greets, Marko (who is now eventually truly off for tonight) Visit http://mail.kde.org/mailman/listinfo/kde-devel#unsub to unsubscribe
