Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread José Ignacio
When you implement command line switches you will probably want --help to emit a translated message. On Fri, May 3, 2019 at 1:51 PM Wayne Stambaugh wrote: > On 5/3/2019 11:27 AM, Dick Hollenbeck wrote: > > On 5/3/19 9:41 AM, Wayne Stambaugh wrote: > >> There is a secondary goal of removing

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Wayne Stambaugh
On 5/3/2019 11:27 AM, Dick Hollenbeck wrote: > On 5/3/19 9:41 AM, Wayne Stambaugh wrote: >> There is a secondary goal of removing wxWidgets from our low level >> objects. Maybe some day we can build the low level KiCad non-ui >> libraries sans wxWdigets. My thinking is that wxString should only

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Adam Wolf
Yes, we build wxwidgets ourselves! Let me know what settings I should set, and we can get test builds out shortly! https://github.com/KiCad/kicad-mac-builder/blob/master/kicad-mac-builder/wxpython.cmake Adam On Fri, May 3, 2019 at 11:52 AM Wayne Stambaugh wrote: > > It seems like this is

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Dick Hollenbeck
It takes some lead to hit a moving target. You know this if you have ever shot clay pigeons. Do we have any evidence that computers are going to have more memory in the future? If so, then this class might be useful: http://www.cplusplus.com/reference/string/u32string/ I actually don't

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Wayne Stambaugh
It seems like this is macos specific. Don't we have our own custom wxwidgets builds for macos? Maybe we should change the build flags. On 5/3/2019 12:34 PM, Jeff Young wrote: > This is the only bug I could find in the database: > https://bugs.launchpad.net/kicad/+bug/1822678. Note that the

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Jeff Young
This is the only bug I could find in the database: https://bugs.launchpad.net/kicad/+bug/1822678. Note that the Michaels (both Geselbracht and Kavanagh) are on OSX. There’s also the two I fixed (neither of which was logged as I couldn’t figure out how to reproduce them). Once again, OSX.

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Wayne Stambaugh
On 5/3/2019 10:49 AM, Jeff Young wrote: > @Wayne, are you sure about those settings?  (That’s the flag I proposed > we flip in the previous incantation of this thread.)  I’m fairly sure > (although not 100% certain) that the bug only exists in > wxUSE_UNICODE_UTF8 mode. If that is the case, could

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Jeff Young
I did a bit more sleuthing. Turns out wxUSE_UNICODE_UTF8 is on for OSX. I misread it because wxUSE_STRING_POS_CACHE is Linux-only: #if wxUSE_UNICODE_UTF8 && !defined(__WINDOWS__) && !defined(__WXOSX__) #define wxUSE_STRING_POS_CACHE 1 #else #define wxUSE_STRING_POS_CACHE 0 #endif … and

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Dick Hollenbeck
On 5/3/19 9:41 AM, Wayne Stambaugh wrote: > There is a secondary goal of removing wxWidgets from our low level > objects. Maybe some day we can build the low level KiCad non-ui > libraries sans wxWdigets. My thinking is that wxString should only come > into play at the UI level when dealing with

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Dick Hollenbeck
Could that damage have already been done by prior concurrent access? Maybe a subsequent read only operation fails because your linked list is already hay wired... Walking a linked list at a later point in time than when it was damaged can do this.. On 5/3/19 10:18 AM, Jeff Young wrote: > I

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Jon Evans
I have not yet had a chance to flip any switches anywhere. I could never reproduce these string crashes locally on my Linux machine, I just relied on others' reports. On Fri, May 3, 2019 at 11:16 AM Jeff Young wrote: > More and more strange by the day. My OSX build also shows that >

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Jeff Young
I don’t believe that’s the case. Neither of the two crashes that I tracked down involved direct index access (or any change to the string). One was calling foo.IsEmpty(), and the other foo.Length(). Both use const iterators under the hood. When wxUSE_UNICODE_UTF8 is off, wxWidgets gets

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Dick Hollenbeck
John I got this too from reading the class documentation an hour ago. To smoke these out, a person could comment out the undesirable calls in a wx header, perhaps one that was temporarily moved into a place at a higher priority in the INCLUDE file search space. Then "make -i" perhaps on a

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread John Beard
Hi Jeff, I think it is the index access operator that performs this caching, to allow you to access the n'th code point any number of times while only iterating the string only once. However, you can still use the iterator access safely. It is only index based access that is cached and

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Jeff Young
More and more strange by the day. My OSX build also shows that wxUSE_UNICODE_UTF8 is off (and thereby wxUSE_STRING_POS_CACHE). Or at least CLion thinks it’s off (and highlights the code accordingly). I had earlier encouraged Jon to flip the switch to see what happened. I had meant locally,

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Wayne Stambaugh
On 5/3/19 5:22 AM, Jeff Young wrote: > Hi Dick, > >>> h) What is the list of deficiencies with current string usage? > > I only have one issue with the current use of wxString, but it’s a big > one: it crashes (unpredictably) when used multi-threaded in UTF8 mode. I thought it was wxString

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Jeff Young
Yes, we know exactly why it crashes: in order to speed up iterator access each iterator keeps a pointer into the last location accessed (so that i-1 and i+1 can be fast). These pointers are kept in a linked-list. Adding and removing pointers from this list is not thread-protected. Note that

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Dick Hollenbeck
Thanks Wayne for giving me a chance to participate. Jeff has been very helpful so far. Before we setup a small group communications mechanism, I look forward to Jeff's further input. I think the needs analysis is important before we build a solution work environment. Maybe there's a simple

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Dick Hollenbeck
Thanks Jeff. On 5/3/19 4:22 AM, Jeff Young wrote: > Hi Dick, > >>> h) What is the list of deficiencies with current string usage? > > I only have one issue with the current use of wxString, but it’s a big one: > it crashes > (unpredictably) when used multi-threaded in UTF8 mode. The fact that

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Wayne Stambaugh
Dick, On 5/2/19 6:32 PM, Dick Hollenbeck wrote: > On 4/30/19 4:36 AM, Jeff Young wrote: >> We had talked earlier about throwing the wxWidgets UTF8 compile switch to >> get rid of our wxString re-entrancy problems. However, I noticed that the >> 6.0 work packages doc includes an item for

Re: [Kicad-developers] 6.0 string proposal

2019-05-03 Thread Jeff Young
Hi Dick, >> h) What is the list of deficiencies with current string usage? I only have one issue with the current use of wxString, but it’s a big one: it crashes (unpredictably) when used multi-threaded in UTF8 mode. This design document makes for fascinating reading:

Re: [Kicad-developers] 6.0 string proposal

2019-05-02 Thread Dick Hollenbeck
On 4/30/19 4:36 AM, Jeff Young wrote: > We had talked earlier about throwing the wxWidgets UTF8 compile switch to get > rid of our wxString re-entrancy problems. However, I noticed that the 6.0 > work packages doc includes an item for std::string-ization of the BOARD. > (While a lot more

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jon Evans
String access is a factor in the performance of the new real-time connectivity algorithm in eeschema, since all connectivity is established by parsing labels and pin names. I have not done benchmarks comparing various options for string storage, but we would need to watch that space too if we

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread John Beard
On 30/04/2019 21:55, Jeff Young wrote: I think we’re a long way from handling Hiragana, Katakana or Kanji. Probably the same for Tamil or Telegu, although I know nothing about them. We can already accept Unicode from many string user inputs. For example a net name can be "" (this is 2

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jeff Young
I think we’re a long way from handling Hiragana, Katakana or Kanji. Probably the same for Tamil or Telegu, although I know nothing about them. So why do we scan strings? We do it when tokenizing, but all our tokens are roman (if not ascii), so that should be OK. We also do it looking for

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread John Beard
On 30/04/2019 18:19, Jeff Young wrote: I was referring to UCS-2 or UCS-4. I’m evidently behind the times, though, because I now see that UTF-32 and UCS-4 are equivalent. (Which means that both some of John’s original premises and my quote in teal below were wrong: UTF32 is indeed a one:one

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jeff Young
I was referring to UCS-2 or UCS-4. I’m evidently behind the times, though, because I now see that UTF-32 and UCS-4 are equivalent. (Which means that both some of John’s original premises and my quote in teal below were wrong: UTF32 is indeed a one:one map between code points and chars.) So my

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Seth Hillbrand
Am 2019-04-30 12:49, schrieb Jeff Young: You are correct that you also can’t do it with UTF32 strings, but I’m not suggesting those. I’m suggesting *unicode* strings. That’s 1 code-point per character. So myString[3] still works. Sorry Jeff, I'm being slow here and must be missing an

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jeff Young
Thanks for the analysis, John. However, those numbers are with wxWidget’s performance optimizations (the ones that crash when multi-threaded), so we don’t really know how bad they would be without it. Also, wxString hides the UTF8 serialization from us and makes for simpler code. You can use

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Seth Hillbrand
Am 2019-04-30 12:22, schrieb John Beard: Even with UTF-32, you can only do an O(1) lookup of the n'th *code point* or *code unit* (the same in UTF-32, not in UTF-8), not the n'th *encoded character*. +1 here. I'd be in favor of standardizing on a clean, standards-compliant string library for

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread John Beard
On 30/04/2019 16:01, Jeff Young wrote: Primarily for performance reasons. WRT performance, I did a few benchmarks for reference (on Linux) Loading this large CIAA PCB[1] allocates, out of a peak usage of 467MB of heap with a 0.01% threshold: * 9.6MB of std::basic_string::_M_assign *

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jeff Young
Primarily for performance reasons. Because characters are different lengths, you have to scan the string to find the n’th character. (wxString solves this with cached iterators, but they’re not thread-safe and so we get random crashes from them.) > On 30 Apr 2019, at 15:59, Dmitry Salychev

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Dmitry Salychev
On Tue, Apr 30, 2019 at 08:59:46AM -0400, Wayne Stambaugh wrote: > Given that std::wstring is platform dependent, I would be opposed to > using it. I'm not opposed to std::u32string but UTF8 is pretty well > vetted so please keep that in mind. I think the possibility of breakage > is low but I'm

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jeff Young
The GUI is strongly bound to wxWidgets anyway, so I don’t have an issue using wxString there. Although I’m sure we could create our own “_()” macro at some point if it comes to that. > On 30 Apr 2019, at 14:55, Wayne Stambaugh wrote: > > What about translated strings? Is changing them from

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Wayne Stambaugh
What about translated strings? Is changing them from wxString to u32string going to be an issue? I know this doesn't effect the file I/O strings but it is something we are going to have to consider if we are going to punt wxString. On 4/30/19 9:27 AM, Jeff Young wrote: > Sure, but we’re going

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jeff Young
Sure, but we’re going to be re-writing the parsers and formatters for s-expr so it’s going to be all different code anyway. (Granted the new code could have used the old infrastructure, but I think we need to wean ourselves from wxString either way.) > On 30 Apr 2019, at 13:59, Wayne

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Wayne Stambaugh
Given that std::wstring is platform dependent, I would be opposed to using it. I'm not opposed to std::u32string but UTF8 is pretty well vetted so please keep that in mind. I think the possibility of breakage is low but I'm not naive enough to think that it's zero. You would have to do some

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Jeff Young
I suspect all our platforms use at least 32 bit ints, but even so std::u32string does communicate the intent better. So change the proposal to that…. Cheers, Jeff. > On 30 Apr 2019, at 10:52, Andrew Lutsenko wrote: > > Hi, > I have no opinion on the matter but would add a reminder that

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Thomas Pointhuber
An: "Jeff Young" Cc: "KiCad Developers" Betreff: Re: [Kicad-developers] 6.0 string proposal Hi, I have no opinion on the matter but would add a reminder that wchar_t is platform and compiler dependent. Consider using std::u32string instead of std::wstring if you want a

Re: [Kicad-developers] 6.0 string proposal

2019-04-30 Thread Andrew Lutsenko
Hi, I have no opinion on the matter but would add a reminder that wchar_t is platform and compiler dependent. Consider using std::u32string instead of std::wstring if you want all code points to fit into one element. Regards, Andrew On Tue, Apr 30, 2019 at 2:36 AM Jeff Young wrote: > We had