Re: [firebird-support] running Firebird without fbintl
On 10/07/18 10:00, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: > On 06/07/18 18:59, Dimitry Sibiryakov s...@ibphoenix.com > [firebird-support] wrote: >> 06.07.2018 10:45, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] >> wrote: With GCC "-fvisibility=internal" can be used to suppress exporting of all symbols. >>> We currently use -fvisibility-inlines-hidden and -fvisibility=hidden for >>> clang... >> But still framework somehow calls internal delete. Did you check >> exported symbols with >> nm (or whatever utility on MacOS can show public symbols of a library)? > Right, it looks like fbintl is exporting operator delete() and operator > delete[](), while libfbembed.dylib is not. > The darwin makefile failed to pass the exported symbols list to the linker for fbintl. It was correct for the other libraries, and it looks like this was already fixed in v3. I rebuilt locally with that fixed and my problem is solved. I reported it as CORE-5868. Hamish ++ Visit http://www.firebirdsql.org and click the Documentation item on the main (top) menu. Try FAQ and other links from the left-side menu there. Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ ++ Yahoo Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/firebird-support/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/firebird-support/join (Yahoo! ID required) <*> To change settings via email: firebird-support-dig...@yahoogroups.com firebird-support-fullfeatu...@yahoogroups.com <*> To unsubscribe from this group, send an email to: firebird-support-unsubscr...@yahoogroups.com <*> Your use of Yahoo Groups is subject to: https://info.yahoo.com/legal/us/yahoo/utos/terms/
Re: [firebird-support] running Firebird without fbintl
On 06/07/18 18:59, Dimitry Sibiryakov s...@ibphoenix.com [firebird-support] wrote: > 06.07.2018 10:45, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] > wrote: >>> With GCC "-fvisibility=internal" can be used to suppress exporting of >>> all symbols. >> We currently use -fvisibility-inlines-hidden and -fvisibility=hidden for >> clang... > But still framework somehow calls internal delete. Did you check exported > symbols with > nm (or whatever utility on MacOS can show public symbols of a library)? Right, it looks like fbintl is exporting operator delete() and operator delete[](), while libfbembed.dylib is not. macOS has nm but it doesn't demangle. They have their own utility symbols(1). Here's the whole list of symbols exported (flags including "EXT") from fbintl: $ symbols Versions/A/fbintl| grep " EXT" 0x16c0 ( 0x130) LD_lookup_charset [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x1840 ( 0x340) LD_lookup_texttype [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x1b80 ( 0x2a0) LD_setup_attributes [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x1ef0 ( 0x40) LD_version [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x00037b90 ( 0x40) operator new(unsigned long) [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x00037c20 ( 0x40) operator new[](unsigned long) [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x00037c60 ( 0x40) operator delete(void*) [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x00037cf0 ( 0x40) operator delete[](void*) [FUNC, EXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] ie new, new[], delete and delete[] are exported. Meanwhile libfbembed.dylib (ie the main Firebird binary) has these marked as PEXT = private-external. $ symbols libfbembed.dylib | grep delete . 0x30451b70 ( 0x40) operator delete(void*) [FUNC, PEXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] 0x30451bb0 ( 0x40) operator delete[](void*) [FUNC, PEXT, NameNList, MangledNameNList, Merged, NList, FunctionStarts] Now to work out how to fix it. Hamish ++ Visit http://www.firebirdsql.org and click the Documentation item on the main (top) menu. Try FAQ and other links from the left-side menu there. Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ ++ Yahoo Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/firebird-support/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/firebird-support/join (Yahoo! ID required) <*> To change settings via email: firebird-support-dig...@yahoogroups.com firebird-support-fullfeatu...@yahoogroups.com <*> To unsubscribe from this group, send an email to: firebird-support-unsubscr...@yahoogroups.com <*> Your use of Yahoo Groups is subject to: https://info.yahoo.com/legal/us/yahoo/utos/terms/
Re: [firebird-support] running Firebird without fbintl
On 06/07/18 20:15, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: On 06/07/18 18:50, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: On 06/07/18 18:20, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] wrote:> 2.5.8 64bit Firebird now compiles with -O0 because of a problem with optimisation in fbintl that caused Firebird to crash... I have not seen any crashes with Firebird, but I will compile up 2.5.8 and see if it helps. Same problem with 2.5.8 - delete in fbintl crashed, called from macOS thread. I am using the 2.5.8 binaries from firebirdsql.org (not compiling my own). Thread 29 Crashed:: Dispatch queue: NSTextCheckingOperationQueue (QOS: USER_INTERACTIVE) 0 libsystem_kernel.dylib 0x7fff5fa30b6e __pthread_kill + 10 1 libsystem_pthread.dylib 0x7fff5fbfb080 pthread_kill + 333 2 libsystem_c.dylib 0x7fff5f98c1ae abort + 127 3 libc++abi.dylib 0x7fff5d88af8f abort_message + 245 4 libc++abi.dylib 0x7fff5d88b12b default_terminate_handler() + 265 5 libobjc.A.dylib 0x7fff5eccba4f _objc_terminate() + 105 6 libc++abi.dylib 0x7fff5d8a67c9 std::__terminate(void (*)()) + 8 7 libc++abi.dylib 0x7fff5d8a6843 std::terminate() + 51 8 libc++abi.dylib 0x7fff5d8a6794 std::__unexpected(void (*)()) + 6 9 libc++abi.dylib 0x7fff5d8a6fd7 __cxa_call_unexpected + 137 10 fbintl 0x000117dc5c92 operator delete(void*) + 50 11 com.apple.CoreNLP 0x7fff4b0307a3 CoreNLP::ICUTextBreakWithCustomizedRules::ICUTextBreakWithCustomizedRules(CoreNLP::TaggerContext const&, __CFURL const*, bool, bool, void (CFRange, bool&, void (NLToken*, bool*) block_pointer) block_pointer) + 323 12 com.apple.CoreNLP 0x7fff4b026f54 CoreNLP::DefaultSubWordTagger::DefaultSubWordTagger(CoreNLP::TaggerContext const&, CoreNLP::StringBuffer const&, unsigned long, __CFURL const*) + 282 13 com.apple.CoreNLP 0x7fff4aff0b7e CoreNLP::WordDispatchTagger::WordDispatchTagger(CoreNLP::TaggerContext&, __CFURL const*) + 220 14 com.apple.CoreNLP 0x7fff4b03182d CoreNLP::WordTagger::createWordTagger(CoreNLP::TaggerContext&) + 41 15 com.apple.CoreNLP 0x7fff4b003b24 NLTokenizerCreate + 100 16 com.apple.CoreNLP 0x7fff4aff5e50 NLStringTokenizerCreate + 388 17 com.apple.CoreFoundation 0x7fff37a8e6d2 CFStringTokenizerCreate + 146 18 com.apple.AppKit 0x7fff359adb6a scoreParagraph + 185 19 com.apple.AppKit 0x7fff3516f680 NSSpellCheckerCheckString + 3952 20 com.apple.AppKit 0x7fff3525e52d -[NSTextCheckingOperation main] + 151 21 com.apple.Foundation 0x7fff39b6a8ee -[__NSOperationInternal _start:] + 778 22 com.apple.Foundation 0x7fff39b66917 __NSOQSchedule_f + 369 23 libdispatch.dylib 0x7fff5f8a6db8 _dispatch_client_callout + 8 24 libdispatch.dylib 0x7fff5f8b9e81 _dispatch_continuation_pop + 472 25 libdispatch.dylib 0x7fff5f8b1733 _dispatch_async_redirect_invoke + 703 26 libdispatch.dylib 0x7fff5f8a89a9 _dispatch_root_queue_drain + 515 27 libdispatch.dylib 0x7fff5f8a8755 _dispatch_worker_thread3 + 101 28 libsystem_pthread.dylib 0x7fff5fbf8169 _pthread_wqthread + 1387 29 libsystem_pthread.dylib 0x7fff5fbf7be9 start_wqthread + 13 Hamish
Re: [firebird-support] running Firebird without fbintl
On 06/07/18 18:50, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: On 06/07/18 18:20, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] wrote:> 2.5.8 64bit Firebird now compiles with -O0 because of a problem with optimisation in fbintl that caused Firebird to crash... I have not seen any crashes with Firebird, but I will compile up 2.5.8 and see if it helps. Same problem with 2.5.8 - delete in fbintl crashed, called from macOS thread. I am using the 2.5.8 binaries from firebirdsql.org (not compiling my own). Hamish
Re: [firebird-support] running Firebird without fbintl
06.07.2018 10:45, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] wrote: >> With GCC "-fvisibility=internal" can be used to suppress exporting of >> all symbols. > We currently use -fvisibility-inlines-hidden and -fvisibility=hidden for > clang... But still framework somehow calls internal delete. Did you check exported symbols with nm (or whatever utility on MacOS can show public symbols of a library)? -- WBR, SD. ++ Visit http://www.firebirdsql.org and click the Documentation item on the main (top) menu. Try FAQ and other links from the left-side menu there. Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ ++ Yahoo Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/firebird-support/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/firebird-support/join (Yahoo! ID required) <*> To change settings via email: firebird-support-dig...@yahoogroups.com firebird-support-fullfeatu...@yahoogroups.com <*> To unsubscribe from this group, send an email to: firebird-support-unsubscr...@yahoogroups.com <*> Your use of Yahoo Groups is subject to: https://info.yahoo.com/legal/us/yahoo/utos/terms/
Re: [firebird-support] running Firebird without fbintl
On 06/07/18 18:20, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] wrote:> 2.5.8 64bit Firebird now compiles with -O0 because of a problem with optimisation in fbintl that caused Firebird to crash... I have not seen any crashes with Firebird, but I will compile up 2.5.8 and see if it helps. Thanks, Hamish
RE: [firebird-support] running Firebird without fbintl
>With GCC "-fvisibility=internal" can be used to suppress exporting of all > symbols. We currently use -fvisibility-inlines-hidden and -fvisibility=hidden for clang... Paul > ++ > > Visit http://www.firebirdsql.org and click the Documentation item > on the main (top) menu. Try FAQ and other links from the left-side menu > there. > > Also search the knowledgebases at > http://www.ibphoenix.com/resources/documents/ > > ++ > > > Yahoo Groups Links > > > > ++ Visit http://www.firebirdsql.org and click the Documentation item on the main (top) menu. Try FAQ and other links from the left-side menu there. Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ ++ Yahoo Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/firebird-support/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/firebird-support/join (Yahoo! ID required) <*> To change settings via email: firebird-support-dig...@yahoogroups.com firebird-support-fullfeatu...@yahoogroups.com <*> To unsubscribe from this group, send an email to: firebird-support-unsubscr...@yahoogroups.com <*> Your use of Yahoo Groups is subject to: https://info.yahoo.com/legal/us/yahoo/utos/terms/
Re: [firebird-support] running Firebird without fbintl
06.07.2018 2:34, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: > Here are a couple of example traces. As you can see neither of these involve > our > application code, they are pure macOS framework threads, except that fbintl > is in the trace. It looks like fbintl exports too much symbols including internal overloaded delete() operator. That triggers well known POSIX "symbol hell". The library must export only public functions. With GCC "-fvisibility=internal" can be used to suppress exporting of all symbols. -- WBR, SD. ++ Visit http://www.firebirdsql.org and click the Documentation item on the main (top) menu. Try FAQ and other links from the left-side menu there. Also search the knowledgebases at http://www.ibphoenix.com/resources/documents/ ++ Yahoo Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/firebird-support/ <*> Your email settings: Individual Email | Traditional <*> To change settings online go to: http://groups.yahoo.com/group/firebird-support/join (Yahoo! ID required) <*> To change settings via email: firebird-support-dig...@yahoogroups.com firebird-support-fullfeatu...@yahoogroups.com <*> To unsubscribe from this group, send an email to: firebird-support-unsubscr...@yahoogroups.com <*> Your use of Yahoo Groups is subject to: https://info.yahoo.com/legal/us/yahoo/utos/terms/
RE: [firebird-support] running Firebird without fbintl
<> 2.5.8 64bit Firebird now compiles with -O0 because of a problem with optimisation in fbintl that caused Firebird to crash... Paul
Re: [firebird-support] running Firebird without fbintl
On 2018-07-06 02:07, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: > On 05/07/18 21:06, Mark Rotteveel m...@lawinegevaar.nl > [firebird-support] wrote: > >> On 5-7-2018 09:28, Hamish Moffatt ham...@risingsoftware.com >> [firebird-support] wrote: >> >>> So, what is the impact of removing fbintl? What functionality do I >> lose? >> >> You lose almost all character set support, including UTF-8. > > Right. Meaning ORDER BY character fields as well as functions like > substring(), char_length()? > > In the absence of fbintl does Firebird interpret those fields just as > pure ASCII/Latin1 then? No, you'll receive an error that the character set has not been found. Falling back to another character set would be a great way to get logical or even physical corruption of data. Mark
Re: [firebird-support] running Firebird without fbintl
On 06/07/18 10:34, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: We are using Firebird via Qt. Qt dynamically loads its Firebird driver (libqsqlibase.dylib) which is linked against the Firebird framework which is dynamically loading fbintl. Actually, this isn't the whole story. We are also statically linking IBPP and linking the Firebird framework directly for that. As well it is loaded by the Qt driver. Hamish
Re: [firebird-support] running Firebird without fbintl
On 05/07/18 22:52, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] wrote: And a couple of final questions, what optimisation level are you using during the build? 32bit or 64bit Firebird? 64-bit. I have not changed any optimisation flags so it looks like it's being compiled with -O1 (from builds/posix/prefix.darwin_x86_64). Hamish
Re: [firebird-support] running Firebird without fbintl
On 05/07/18 22:47, 'Paul Beach' pbe...@mail.ibphoenix.com [firebird-support] wrote: <> MacOS does not use a full icu implementation, so I doubt you are linking against the system icu lib (/usr/lib/libicucore.dylib) unless of course you are specifically requesting it. To be honest I doubt you could build Firebird if you hadn't installed a proper copy of icu from elsewhere. Personally I expect the problem is elsewhere... Does it crash on 10.13 or on earlier versions of the OS? Have you analysed a crash dump? What happens if you use isql on your embedded version of Firebird? What version of Firebird are you using? It only crashes on 10.13, not earlier versions. (I haven't tried the 10.14 beta yet.) I'm using Firebird 2.5.7. Looking at the traces again icu is probably not relevant at all actually. (The Firebird sources includes icu 30 in extern, so I did not have to install my own.) Firebird works fine, but invoking the Mac file open dialog causes the application to crash, and the back trace always references fbintl. We are using Firebird via Qt. Qt dynamically loads its Firebird driver (libqsqlibase.dylib) which is linked against the Firebird framework which is dynamically loading fbintl. It looks like the custom memory allocator that fbintl is using is involved. Perhaps it didn't get initialized properly because of the dynamic loading of the Firebird framework etc? Here are a couple of example traces. As you can see neither of these involve our application code, they are pure macOS framework threads, except that fbintl is in the trace. Thread 6 Crashed:: Dispatch queue: quicklook.pluginload 0 libsystem_pthread.dylib 0x7fff69cd73d5 pthread_mutex_lock + 0 1 fbintl 0x000115339889 Firebird::Mutex::enter() + 9 2 fbintl 0x000115341c9e Firebird::MemoryPool::deallocate(void*) + 62 3 fbintl 0x000115342279 operator delete(void*) + 9 4 com.apple.security.csparser 0x00011b698de9 Security::CodeSigning::Requirement const* Security::CodeSigning::parsestd::__1::basic_string, std::__1::allocator >, Security::CodeSigning::Requirement>(std::__1::basic_stringstd::__1::char_traits, std::__1::allocator >, Security::CodeSigning::Requirement* (Security_CodeSigning::RequirementParser::*)(), std::__1::basic_string, std::__1::allocator >&) + 258 5 com.apple.security.csparser 0x00011b698166 Security::CodeSigning::stringRequirement(std::__1::basic_stringstd::__1::char_traits, std::__1::allocator >, std::__1::basic_string, std::__1::allocator >&) + 52 6 com.apple.security 0x7fff4de32dd8 Security::CodeSigning::Requirement const* Security::CodeSigning::parsestd::__1::basic_string, std::__1::allocator > >(std::__1::basic_stringstd::__1::char_traits, std::__1::allocator >, Security::CodeSigning::Requirement const* (* Security::CodeSigning::AntlrPlugin::*)(std::__1::basic_stringstd::__1::char_traits, std::__1::allocator >, std::__1::basic_string, std::__1::allocator >&)) + 74 7 com.apple.security 0x7fff4de32d58 Security::CodeSigning::RequirementParser::operator()(std::__1::basic_stringstd::__1::char_traits, std::__1::allocator > const&) + 34 8 com.apple.security 0x7fff4de06234 SecRequirementCreateWithStringAndErrors + 70 9 com.apple.QuickLookFramework 0x7fff4d4dc76d QLCheckAppleSignature + 69 10 com.apple.QuickLookFramework 0x7fff4d4a7fc4 _QLLoadPluginAtURL + 1267 11 com.apple.QuickLookFramework 0x7fff4d4a74ec ___QLLaunchUpdatingThread_block_invoke + 1543 12 libdispatch.dylib 0x7fff69a1c591 _dispatch_call_block_and_release + 12 13 libdispatch.dylib 0x7fff69a14d50 _dispatch_client_callout + 8 14 libdispatch.dylib 0x7fff69a2920c _dispatch_queue_serial_drain + 635 15 libdispatch.dylib 0x7fff69a1c0fd _dispatch_queue_invoke + 373 16 libdispatch.dylib 0x7fff69a29f02 _dispatch_root_queue_drain_deferred_wlh + 332 17 libdispatch.dylib 0x7fff69a2dd16 _dispatch_workloop_worker_thread + 880 18 libsystem_pthread.dylib 0x7fff69cd9033 _pthread_wqthread + 980 19 libsystem_pthread.dylib 0x7fff69cd8c4d start_wqthread + 13 Thread 1 Crashed:: Dispatch queue: NSTextCheckingOperationQueue (QOS: USER_INTERACTIVE) 0 fbintl 0x00010f3ebcd8 Firebird::MemoryPool::deallocate(void*) + 120 1 fbintl 0x00010f3ec279 operator delete(void*) + 9 2 com.apple.CoreNLP 0x7fff3d446b03 CoreNLP::ICUTextBreakWithCustomizedRules::ICUTextBreakWithCustomizedRules(CoreNLP::TaggerContext const&, __CFURL const*, bool, bool, void (CFRange, bool&, void (NLToken*, bool*) block_pointer) block_pointer) + 323 3 com.apple.CoreNLP
Re: [firebird-support] running Firebird without fbintl
On 05/07/18 21:06, Mark Rotteveel m...@lawinegevaar.nl [firebird-support] wrote: On 5-7-2018 09:28, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: > So, what is the impact of removing fbintl? What functionality do I lose? You lose almost all character set support, including UTF-8. Right. Meaning ORDER BY character fields as well as functions like substring(), char_length()? In the absence of fbintl does Firebird interpret those fields just as pure ASCII/Latin1 then? Hamish
RE: [firebird-support] running Firebird without fbintl
And a couple of final questions, what optimisation level are you using during the build? 32bit or 64bit Firebird? Paul -Original Message- From: firebird-support@yahoogroups.com [mailto:firebird-supp...@yahoogroups..com] Sent: 05 July 2018 14:47 To: firebird-support@yahoogroups.com Subject: RE: [firebird-support] running Firebird without fbintl <> MacOS does not use a full icu implementation, so I doubt you are linking against the system icu lib (/usr/lib/libicucore.dylib) unless of course you are specifically requesting it. To be honest I doubt you could build Firebird if you hadn't installed a proper copy of icu from elsewhere. Personally I expect the problem is elsewhere... Does it crash on 10.13 or on earlier versions of the OS? Have you analysed a crash dump? What happens if you use isql on your embedded version of Firebird? What version of Firebird are you using? Paul
RE: [firebird-support] running Firebird without fbintl
<> MacOS does not use a full icu implementation, so I doubt you are linking against the system icu lib (/usr/lib/libicucore.dylib) unless of course you are specifically requesting it. To be honest I doubt you could build Firebird if you hadn't installed a proper copy of icu from elsewhere. Personally I expect the problem is elsewhere... Does it crash on 10.13 or on earlier versions of the OS? Have you analysed a crash dump? What happens if you use isql on your embedded version of Firebird? What version of Firebird are you using? Paul
Re: [firebird-support] running Firebird without fbintl
On 5-7-2018 09:28, Hamish Moffatt ham...@risingsoftware.com [firebird-support] wrote: > We're linking embedded Firebird in to our application on macOS. Ever > since macOS 10.13, we are getting crashes whenever our application > invokes a file open dialog. > > We have tracked this down to some sort of conflict with fbintl. If we > remove fbintl from our application, everything works. > > I suspect it is some sort of conflict between libicu linked into fbintl > and libicu used in the macOS system frameworks (and I suspect a change > to icu in macOS is why this only started with 10.13). I tried to rebuild > Firebird with a later version but didn't get very far. > > > So, what is the impact of removing fbintl? What functionality do I lose? You lose almost all character set support, including UTF-8. Mark -- Mark Rotteveel
[firebird-support] running Firebird without fbintl
We're linking embedded Firebird in to our application on macOS. Ever since macOS 10.13, we are getting crashes whenever our application invokes a file open dialog. We have tracked this down to some sort of conflict with fbintl. If we remove fbintl from our application, everything works. I suspect it is some sort of conflict between libicu linked into fbintl and libicu used in the macOS system frameworks (and I suspect a change to icu in macOS is why this only started with 10.13). I tried to rebuild Firebird with a later version but didn't get very far. So, what is the impact of removing fbintl? What functionality do I lose? Hamish
