Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-30 Thread Jonas Maebe
On 30 Jul 2013, at 12:04, Noah Silva wrote: It causes the compiler to interpret the string constants in your program as UTF-8 rather than as unknown binary data, and hence convert them at run time to the current ansi code page when assigning them to an ansistring/shortstring. This is unrela

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-30 Thread Noah Silva
Hi, > > >> If your source code is in UTF8 but you do not tell this to the compiler, > ... > > tried that some of the time. I have never had any problems other than > the > > Windows Console issue. > > As mentioned before, that causes the compiler to directly pass your UTF-8 > data around with a

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-30 Thread Jonas Maebe
On 30 Jul 2013, at 04:17, Noah Silva wrote: > 2013/7/29 Jonas Maebe > >> If your source code is in UTF8 but you do not tell this to the compiler, >> it will directly pass the "garbage" in your source code to the Win32 APIs. >> After all, there is no way for it to know what else it should with i

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-30 Thread Michael Schnell
On 07/30/2013 04:29 AM, Noah Silva wrote: No, UTF16 only needs more memory if most of the text is ASCII. It actually uses less than UTF8 in the average case for Japanese, for example. Of course you are right here. Linux OS API in most cases is 8 Bit, I assume by 8bit, you mean variabl

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-29 Thread Noah Silva
Hi, 2013/7/29 Michael Schnell > On 07/29/2013 07:36 AM, Noah Silva wrote: > > >> Using UTF16 for internal string handling is a sensible option. > > It depends. > UTF-16 needs more memory used > No, UTF16 only needs more memory if most of the text is ASCII. It actually uses less than UTF8 in

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-29 Thread Noah Silva
Hi, 2013/7/29 Jonas Maebe > > On 29 Jul 2013, at 08:01, Noah Silva wrote: > > > Oddly enough, I can get my (2.6.1) version to change the garbage it shows > > but but saving the file as UTF8 with BOM. (Until now, I had been using > > UTF8 w/o BOM). > > If your source code is in UTF8 but you do n

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-29 Thread Michael Schnell
On 07/29/2013 12:12 PM, Mattias Gaertner wrote: fpc is not "moving" to Delphi strings. That would break all compatibility. This is exactly what Delphi did :-( . It will support both. There might be an appropriate compiler switch. But what about the RTL-Interface like the type StringList offe

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-29 Thread Mattias Gaertner
On Mon, 29 Jul 2013 12:00:34 +0200 Michael Schnell wrote: >[...] > > To address your specific points: > > 1.Lazarus User API already supports UTF8 so far as I know. Yes, since many years. > I suppose this is bound to change once fpc has completed the move to > "new Delphi Strings". fpc is not

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-29 Thread Michael Schnell
On 07/29/2013 07:36 AM, Noah Silva wrote: Using UTF16 for internal string handling is a sensible option. It depends. UTF-16 needs more memory used Linux OS API in most cases is 8 Bit, while Windows OS API is 16 bit Conversions are very expensive. If you need to import export much data but don'

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-29 Thread Jonas Maebe
On 29 Jul 2013, at 08:01, Noah Silva wrote: > Oddly enough, I can get my (2.6.1) version to change the garbage it shows > but but saving the file as UTF8 with BOM. (Until now, I had been using > UTF8 w/o BOM). If your source code is in UTF8 but you do not tell this to the compiler, it will dir

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-28 Thread Noah Silva
Hi, 2013/7/9 Jonas Maebe > > The console code page. You can get it using the following function: > function GetConsoleOutputCP : UINT; stdcall; external 'kernel32' name > 'GetConsoleOutputCP'; > > 2. Is there any reason why writing out data in the local encoding (with >> write statements, etc

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-28 Thread Noah Silva
Hi, 2013/7/9 Michael Schnell > On 07/09/2013 11:02 AM, Noah Silva wrote: > >> >> I convert it to UTF8 before displaying it >> >> Not a good idea. > > Well if the console is UTF8 > The FPC developers are right now busy implementing the new Delphi Strings. > This _could_ mean that the a

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-28 Thread Noah Silva
Hi, 2013/7/9 Jonas Maebe > > On 09 Jul 2013, at 11:02, Noah Silva wrote: > > 1. What encoding "should" I be writing to the terminal? >> > > The console code page. You can get it using the following function: > function GetConsoleOutputCP : UINT; stdcall; external 'kernel32' name > 'GetConsole

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-28 Thread Noah Silva
Hello everyone, OK I have checked this out: 2013/7/9 Tomas Hajny > > There is code page 65001 for UTF-8, however I don't know since which > version of MS Windows it is supported (and to which extent - e.g. > including the console output). You may change the output code page for > console using t

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-28 Thread Noah Silva
Hi, This answer is a bit late, but... 2013/7/9 Dennis Poon > > I have followed steps in the Wiki, etc., but to little avail, so I have >> some questions for anyone who knows more than me: >> ... > > ... > > 4. I write the output to a file instead of the console, and it's ok. >> > > Please stat

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-14 Thread Noah Silva
Hi, 2013/7/10 Graeme Geldenhuys > On 2013-07-09 13:08, Tomas Hajny wrote: > >> Afaik Windows XP, but there might be font problems on console. > > I also thought about mentioning True Type fonts (like Lucida) being a > The font is not the issue here... If you see text that looks like lots of e

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-14 Thread Noah Silva
Hi, 2013/7/9 Marco van de Voort > In our previous episode, Dennis Poon said: > > Please state the windows version you are using. XP or Windows 7? > > In XP there were separate Far East versions of Windows. In Vista+ this > was wholly integrated and there is only one Vista base system with vario

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-14 Thread Marco van de Voort
In our previous episode, Noah Silva said: > Using powershell saves me a little sanity since I am used to Unix Systems > and I can't install Cygwin everywhere. > > At any rate, I assume that since the console is a separate thing from the > program running in it that powershell and cmd would be the

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-13 Thread Noah Silva
Hi, Some comments... 2013/7/9 Marco van de Voort > In our previous episode, Tomas Hajny said: > > There is code page 65001 for UTF-8, however I don't know since which > > version of MS Windows it is supported (and to which extent - e.g. > > including the console output). > > Afaik Windows XP, b

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-13 Thread Noah Silva
Hi, First, I want thank everyone for taking the time to answer! 2013/7/9 Jonas Maebe > > On 09 Jul 2013, at 11:02, Noah Silva wrote: > > 1. What encoding "should" I be writing to the terminal? >> > > The console code page. You can get it using the following function: > function GetConsoleOut

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-11 Thread Graeme Geldenhuys
On 2013-07-10 11:19, Tomas Hajny wrote: > > Sure, but you can't select just any True Type fonts for console windows > (only Lucida is offered in WinXP; And is so since Win95. Just amazing that Windows Console is so far behind other platforms - it's rather ridiculous if you think about it. If I u

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-10 Thread Tomas Hajny
On Wed, July 10, 2013 10:30, Graeme Geldenhuys wrote: > On 2013-07-09 13:08, Tomas Hajny wrote: >>> Afaik Windows XP, but there might be font problems on console. >> I also thought about mentioning True Type fonts (like Lucida) being a > > > It's not so much about True Type fonts, but about enablin

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-10 Thread Graeme Geldenhuys
On 2013-07-09 13:08, Tomas Hajny wrote: >> Afaik Windows XP, but there might be font problems on console. > I also thought about mentioning True Type fonts (like Lucida) being a It's not so much about True Type fonts, but about enabling a Unicode font in the console. There are many True Type font

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Tomas Hajny
On Tue, July 9, 2013 14:38, Marco van de Voort wrote: > In our previous episode, Tomas Hajny said: >> >> is >> >> like this. >> > >> > Keep in mind he is talking powershell, not cmd.exe >> >> I have no experience with powershell yet, but as long as it uses a >> console >> window (albeit designed di

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Marco van de Voort
In our previous episode, Tomas Hajny said: > >> is > >> like this. > > > > Keep in mind he is talking powershell, not cmd.exe > > I have no experience with powershell yet, but as long as it uses a console > window (albeit designed differently), the launched applications should > behave the same wa

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Tomas Hajny
On Tue, July 9, 2013 12:40, Marco van de Voort wrote: > In our previous episode, Tomas Hajny said: >> >> and be done with it? >> > >> > I don't know. >> >> There is code page 65001 for UTF-8, however I don't know since which >> version of MS Windows it is supported (and to which extent - e.g. >> in

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Marco van de Voort
In our previous episode, Dennis Poon said: > Please state the windows version you are using. XP or Windows 7? In XP there were separate Far East versions of Windows. In Vista+ this was wholly integrated and there is only one Vista base system with various language packs. It might be that some o

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Michael Schnell
On 07/09/2013 12:55 PM, Marco van de Voort wrote: Well, the main reason is not FPC, but Windows. UTF8 in Windows on _API_ level is simply not a good idea You are absolutely right. Here, in theory the Lazarus LCL and the FPC RTL could help by by providing an application programmer API that doe

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Dennis Poon
I have followed steps in the Wiki, etc., but to little avail, so I have some questions for anyone who knows more than me: 1. What encoding "should" I be writing to the terminal? from experimenting with text files using the cat command in powershell, it seems that local ("ANSI") encoding shoul

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Marco van de Voort
In our previous episode, Michael Schnell said: > Not a good idea. > > The FPC developers are right now busy implementing the new Delphi > Strings. This _could_ mean that the application programmer can use any > encoding (such as multiple different ANSI byte-codes, UTF-8, UTF-16, > ...), but in

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Michael Schnell
On 07/09/2013 11:02 AM, Noah Silva wrote: I convert it to UTF8 before displaying it Not a good idea. The FPC developers are right now busy implementing the new Delphi Strings. This _could_ mean that the application programmer can use any encoding (such as multiple different ANSI byte-c

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Marco van de Voort
In our previous episode, Tomas Hajny said: > >> and be done with it? > > > > I don't know. > > There is code page 65001 for UTF-8, however I don't know since which > version of MS Windows it is supported (and to which extent - e.g. > including the console output). Afaik Windows XP, but there migh

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Tomas Hajny
On Tue, July 9, 2013 11:13, Jonas Maebe wrote: > On 09 Jul 2013, at 11:02, Noah Silva wrote: . . >> 3. Is there a way to set the output to UTF8 so I can just write out >> UTF8 >> and be done with it? > > I don't know. There is code page 65001 for UTF-8, however I don't know since which version o

Re: [fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Jonas Maebe
On 09 Jul 2013, at 11:02, Noah Silva wrote: 1. What encoding "should" I be writing to the terminal? The console code page. You can get it using the following function: function GetConsoleOutputCP : UINT; stdcall; external 'kernel32' name 'GetConsoleOutputCP'; from experimenting with te

[fpc-pascal] Console Encoding in Windows (Local VS. UTF8)

2013-07-09 Thread Noah Silva
Hi, I deal with in Japanese (and sometimes other languages) in a lot of my programs, and nothing I do seems to work consistently on Windows systems. (OS X is no problem). I have followed steps in the Wiki, etc., but to little avail, so I have some questions for anyone who knows more than me: 1.