On Thu, Nov 03, 2005 at 09:17:43PM -0500, Tom Lane wrote: > Gregory Maxwell <[EMAIL PROTECTED]> writes: > > Another way to look at this is in the context of compression: With > > unicode, characters are really 32bit values... But only a small range > > of these values is common. So we store and work with them in a > > compressed format, UTF-8. > > As such it might be more interesting to ask some other questions like: > > are we using the best compression algorithm for the application, and, > > why do we sometimes stack two compression algorithms? > Actually, the real reason we use UTF-8 and not any of the > sorta-fixed-size representations of Unicode is that the backend is by > and large an ASCII, null-terminated-string engine. *All* of the > supported backend encodings are ASCII-superset codes. Making > everything null-safe in order to allow use of UCS2 or UCS4 would be > a huge amount of work, and the benefit is at best questionable.
Perhaps on a side note - my intuition (which sometimes lies) would tell me that, if the above is true, the backend is doing unnecessary copies of read-only data, if only, to insert a '\0' at the end of the strings. Is this true? I'm thinking along the lines of the other threads that speak of PostgreSQL being CPU or I/O bound, not disk bound, for many sorts of operations. Is PostgreSQL unnecessary copying string data around (and other data, I would assume). Cheers, mark -- [EMAIL PROTECTED] / [EMAIL PROTECTED] / [EMAIL PROTECTED] __________________________ . . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder |\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ | | | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada One ring to rule them all, one ring to find them, one ring to bring them all and in the darkness bind them... http://mark.mielke.cc/ ---------------------------(end of broadcast)--------------------------- TIP 4: Have you searched our list archives? http://archives.postgresql.org