Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-25 Thread Dave Abrahams via swift-evolution
on Sun Aug 14 2016, Michael Savich wrote: > Back in Swift 1.0, subscripting a String was easy, you could just use > subscripting in a very Python like way. Just to correct the record: no, it was really never like that in Swift. -- -Dave ___ swift-

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-22 Thread Félix Cloutier via swift-evolution
I generally agree with Xiaodi. I'm not saying that you need Unicode in your scientific application, but it's very easy to think that you don't need it when you actually do, especially because you don't necessarily get to test yourself. The argument that one doesn't need Unicode is the slippery s

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-22 Thread Charles Srstka via swift-evolution
> On Aug 19, 2016, at 1:05 AM, Félix Cloutier via swift-evolution > wrote: > > Even UTF-32 does not provide a 1-to-1 mapping to visual glyphs. As mentioned > earlier in this thread, for instance, flags are composed of two Unicode > characters. > > Félix I wonder how possible it would be to m

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-22 Thread Xiaodi Wu via swift-evolution
On Sun, Aug 21, 2016 at 3:04 PM, Richard Ward via swift-evolution < swift-evolution@swift.org> wrote: > First, this is my first post to a list like this and I could not find the > instructions to properly respond to a post in the digest. Does one have to > subscribe to the verbose (non digest pos

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-22 Thread Magnus Ahltorp via swift-evolution
> 18 Aug. 2016 07:40 Félix Cloutier via swift-evolution > wrote: > >> In Félix’s case, I would expect to have to ask for a mail-friendly >> representation of his name, just like you have to ask for a >> filesystem-friendly representation of a filename regardless of what the >> internal repres

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-22 Thread Richard Ward via swift-evolution
First, this is my first post to a list like this and I could not find the instructions to properly respond to a post in the digest. Does one have to subscribe to the verbose (non digest post) in order to respond to a thread correctly? Or is there a link to some instructions? Thanks. I come

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-18 Thread Félix Cloutier via swift-evolution
Even UTF-32 does not provide a 1-to-1 mapping to visual glyphs. As mentioned earlier in this thread, for instance, flags are composed of two Unicode characters. Félix > Le 18 août 2016 à 12:22:10, Jean-Denis Muys a écrit : > > And both are variable-length encoding. I mean that different chara

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-18 Thread Félix Cloutier via swift-evolution
When I say "reinterpret", I mean taking the UTF-8 bytes and pretend that they're UTF-16. This is an extremely clear bug whenever it happens. The correct conversion between UTF-8 and UTF-16 is lossless. The vast majority of systems, including file systems and email addresses, support Unicode. I'

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-18 Thread Xiaodi Wu via swift-evolution
Actually, if I'm not mistaken, String (or at least, CFStringRef, to which String is toll-free bridged) does not re-encode anything eagerly. If you initialize with UTF8 bytes, it's stored internally as UTF8 bytes; if you initialize with UTF16 code units, it's stored internally as UTF16 code units. R

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-18 Thread Kenny Leung via swift-evolution
> On Aug 18, 2016, at 11:51 AM, Félix Cloutier wrote: > Of course you'll have problems if you try to interpret UTF-8 as UTF-16 and > vice-versa, but that'll do you regardless of whether you use international > characters or not. This is exactly my point. Even if the internal representation is

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-18 Thread Félix Cloutier via swift-evolution
I'm not sure I understand your comment. UTF-8 and UTF-16 are just two different ways to represent Unicode data, and they can both encode the whole range of Unicode. Of course you'll have problems if you try to interpret UTF-8 as UTF-16 and vice-versa, but that'll do you regardless of whether you

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-18 Thread Kenny Leung via swift-evolution
>> Just because you are using UTF-8 as the internal format, it does not mean >> that universal support is guaranteed. All I meant was this, and nothing more. If the internal format was UTF-8, and you were using a filesystem whose filenames were UTF-16, you would have the same problems. -Kenny

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Félix Cloutier via swift-evolution
> In Félix’s case, I would expect to have to ask for a mail-friendly > representation of his name, just like you have to ask for a > filesystem-friendly representation of a filename regardless of what the > internal representation is. Just because you are using UTF-8 as the internal > format, i

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Xiaodi Wu via swift-evolution
On Wed, Aug 17, 2016 at 5:03 PM, Kenny Leung via swift-evolution < swift-evolution@swift.org> wrote: > > > On Aug 17, 2016, at 1:57 PM, William Sumner > wrote: > > > You may be interested in this article by Mike Ash, which gives a > rationale for the String API, including why indexes aren't simpl

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Kenny Leung via swift-evolution
> On Aug 17, 2016, at 1:57 PM, William Sumner wrote: > You may be interested in this article by Mike Ash, which gives a rationale > for the String API, including why indexes aren't simple integers: > https://www.mikeash.com/pyblog/friday-qa-2015-11-06-why-is-swifts-string-api-so-hard.html Tha

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Kenny Leung via swift-evolution
Looking at the String reference again, I see that Swift.String is subscriptable. Also, I was able to write my “split” function without using subscripting at all: public extension String { public func split(_ pattern :String) -> [String] { var results = [String]() var remainin

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread William Sumner via swift-evolution
> On Aug 17, 2016, at 2:34 PM, Kenny Leung via swift-evolution > wrote: > > >> William Sumner says: >> Can you be more specific about the improvements you’d like to see? Based on >> an earlier message, you want to be able to use subscripting on strings to >> retrieve visual glyphs, but you c

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Kenny Leung via swift-evolution
> William Sumner says: > Can you be more specific about the improvements you’d like to see? Based on > an earlier message, you want to be able to use subscripting on strings to > retrieve visual glyphs, but you can do this now via the .characters property, > which presents a view of the string’

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Shawn Erickson via swift-evolution
I would also like to understand the perceived problem for first time programmers. To me first time programmers would be working with string literals ("hello world"), string literals with values in them ("Hello /(name)"), doing basic string concat, using higher level API of string to do and find thi

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread William Sumner via swift-evolution
> On Aug 17, 2016, at 10:40 AM, Kenny Leung via swift-evolution > wrote: > > I understand that the most friendly approach may not be the most efficient, > but that’s not what I’m pushing for. I’m pushing for "does the thing people > would most commonly expect”. Take a first-time programmer wh

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Shawn Erickson via swift-evolution
I like a "view" based system when looking at a Unicode string. It lets you pick the view of string - defining how it is indexed - based on your needs. A view could be indexed by a human facing glyph, a particular Unicode encoding style, a decompose style, etc. I think that is powerful, useful, and

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Kenny Leung via swift-evolution
> > On Aug 17, 2016, at 12:20 PM, Shawn Erickson wrote: > > As stated earlier it is 2016 I don’t like the tone attached to this statement. > I think the baseline should be robust Unicode support I don’t understand how anything I have pushed for would compromise robust Unicode support. > and

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Sean Heber via swift-evolution
I’m not sure what the current year of the Gregorian calendar has to do with strings. :P l8r Sean > On Aug 17, 2016, at 2:20 PM, Shawn Erickson via swift-evolution > wrote: > > As stated earlier it is 2016, I think the baseline should be robust Unicode > support and what we have in Swift is

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Shawn Erickson via swift-evolution
As stated earlier it is 2016, I think the baseline should be robust Unicode support and what we have in Swift is actually a fairly good way of dealing with it IMHO. I think new to development folks should have this as their baseline as well... not that we shouldn't make it as easy to work with as p

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Kenny Leung via swift-evolution
It seems to me that UTF-8 is the best choice to encode strings in English and English-like character sets for storage, but it’s not clear that it is the most useful or performant internal representation for working with strings. In my opinion, conflating the preferred storage format and the best

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Zach Waldowski via swift-evolution
It's 2016, "the thing people would most commonly expect" impossible-to-screw-up Unicode support that's performance. Optimizing developer experience for beginning developers is just going to lead to software that screws up in situations the developer doesn't anticipate, as F+¬lix notes above. Zacha

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Kenny Leung via swift-evolution
I understand that the most friendly approach may not be the most efficient, but that’s not what I’m pushing for. I’m pushing for "does the thing people would most commonly expect”. Take a first-time programmer who reads any (human) language, and that is what they would expect. Why couldn’t Stri

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-17 Thread Félix Cloutier via swift-evolution
I'd just like to leave it here that Microsoft called me "F+¬lix" in corporate communications this morning. I've never seen that variation before. If Microsoft used Swift, I would like this to be borderline impossible for them to screw up. :) Félix > Le 16 août 2016 à 21:27:54, Xiaodi Wu via sw

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-16 Thread Xiaodi Wu via swift-evolution
Nice, thanks :) FWIW, there are at least some ASCII-specific optimizations internally in String (this was a question asked and not answered in the prior thread). On Tue, Aug 16, 2016 at 11:21 PM, Jacob Bandes-Storch via swift-evolution < swift-evolution@swift.org> wrote: > Here's a little prior d

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-16 Thread Jacob Bandes-Storch via swift-evolution
Here's a little prior discussion about ASCIIString: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151207/002138.html Jacob On Sun, Aug 14, 2016 at 3:41 PM, Michael Savich via swift-evolution < swift-evolution@swift.org> wrote: > Back in Swift 1.0, subscripting a String was easy

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-16 Thread Slava Pestov via swift-evolution
> On Aug 14, 2016, at 3:41 PM, Michael Savich via swift-evolution > wrote: > > What about having an InternalString subclass that only supports one encoding, > allowing it to be subscripted with Ints? The idea is that an InternalString > is for Strings that are more or less hard coded into the

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-16 Thread Xiaodi Wu via swift-evolution
The utf16 property can already be subscripted with an Int, just as you desire, if you import Foundation. (See the code for corelibs-foundation for an intriguing discussion of why you must import Foundation at the moment.) On Tue, Aug 16, 2016 at 02:00 Michael Savich wrote: > What about adding a

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread Félix Cloutier via swift-evolution
The major problem with this approach is that visual glyphs themselves have one level of variable-length encoding, and they sit on top of another variable-length encoding used to represent the Unicode characters (Swift-native Strings are currently encoded as UTF-8). For instance, the visual glyph

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread Xiaodi Wu via swift-evolution
On Mon, Aug 15, 2016 at 1:07 PM, Michael Savich wrote: > At the risk of treading on sacred ground, does Swift need arbitrary > Unicode in identifiers? The Swift guidebook uses emojis as variable names > as an example of the benefits of this which is… not convincing... > The ship has sailed on th

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread William Sumner via swift-evolution
> On Aug 14, 2016, at 4:41 PM, Michael Savich via swift-evolution > wrote: > > Back in Swift 1.0, subscripting a String was easy, you could just use > subscripting in a very Python like way. But now, things are a bit more > complicated. I recognize why we need syntax like str.startIndex.advan

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread Austin Zheng via swift-evolution
I just want to mention that the standard library team (or, more specifically, Dave and Dmitri) is planning a rewrite of Swift's String subsystem, in part to make it easier to work with strings in the "common case". We may want to get their input, or wait until they've prepared a proposal. Best, Au

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread Xiaodi Wu via swift-evolution
Swift supports arbitrary Unicode for identifier names, so Unicode would have to be supported even for debugging strings. On Mon, Aug 15, 2016 at 12:54 Michael Savich wrote: > Well, the thing I've been thinking is that InternalString doesn't have to > be just for learning. There is value in disti

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread Kenny Leung via swift-evolution
I agree with both points of view. I think we need to bring back subscripting on strings which does the thing people would most commonly expect. I would say that the subscripts indexes should correspond to a visual glyph. This seems reasonable to me for most character sets like Roman, Cyrillic,

Re: [swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread Xiaodi Wu via swift-evolution
On Sun, Aug 14, 2016 at 5:41 PM, Michael Savich via swift-evolution < swift-evolution@swift.org> wrote: > Back in Swift 1.0, subscripting a String was easy, you could just use > subscripting in a very Python like way. But now, things are a bit more > complicated. I recognize why we need syntax lik

[swift-evolution] InternalString class for easy String manipulation

2016-08-15 Thread Michael Savich via swift-evolution
Back in Swift 1.0, subscripting a String was easy, you could just use subscripting in a very Python like way. But now, things are a bit more complicated. I recognize why we need syntax like str.startIndex.advancedBy(x) but it has its downsides. Namely, it makes things hard on beginners. If one o