[fpc-devel] FPC 3.0.2 for SPARC
This is something that was discussed on the FPC-Pascal ML but it died. I am able to build installation bundles for SPARC running Linux (Debian) and Solaris (OpenSXCE). The fp IDE works but doesn't have libgdb support, and I've got limited time to struggle up the learning curve. Would these be of use for the downloads area, which at present only has 2.6.2 and 2.4.2 respectively? -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
[fpc-devel] UTF-8 string literals
Hi, AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings instead of UTF8String. Please correct me if I'm wrong. This has several side effects: 1. When using a character outside BMP FPC stops with: Error: UTF-8 code greater than 65535 found For example: const Eyes = '👀'; 2. Assigning a UTF-8 literal to an UTF8String requires a widestringmanager. For example non ISO-8859-1 chars are mangled: var u: UTF8String = 'äöüالعَرَبِيَّة'; 3. PChar on a string literal does not work as expected. You get the bytes of a widestring instead. What would happen if FPC would be extended to store UTF-8 literals as UTF8String? What are the disadvantages? Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] Object checks
Hi Colin, Le 30/04/2017 à 10:53, C Western a écrit : > If I may, a couple of queries about code generated for object checking > when range checks are on (-Cr): > > An explicit check for nil is generated, which seems redundant, as any > attempt to use the value will generate a SIGSEGV anyway on most > platforms, or am I missing something? > > If the fpc_check_object fails it calls RunError, which forces the > application to terminate, whereas a SIGSEGV will raise an exception and > give the user the option of continuing (at least in an LCL app). Is > there any reason why fpc_check_object does not call HandleError? Thanks for the suggestion, I applied a patch in commit 36113 to trunk that substitutes RunError with HandleError. It has been applied to three functions: fpc_help_destructor, fpc_check_object and fpc_check_object_ext Pierre ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPC 3.0.2 for SPARC
Hi Mark, Le 05/05/2017 à 13:00, Mark Morgan Lloyd a écrit : > This is something that was discussed on the FPC-Pascal ML but it died. > > I am able to build installation bundles for SPARC running Linux (Debian) > and Solaris (OpenSXCE). The fp IDE works but doesn't have libgdb > support, and I've got limited time to struggle up the learning curve. > Would these be of use for the downloads area, which at present only has > 2.6.2 and 2.4.2 respectively? Yes it would be very nice, libgdb missing for IDE is really not a big deal... Please let me know how you can send me the files. Thanks again, Pierre ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, 5 May 2017, Mattias Gaertner wrote: Hi, AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings instead of UTF8String. Please correct me if I'm wrong. This has several side effects: 1. When using a character outside BMP FPC stops with: Error: UTF-8 code greater than 65535 found For example: const Eyes = '👀'; 2. Assigning a UTF-8 literal to an UTF8String requires a widestringmanager. For example non ISO-8859-1 chars are mangled: var u: UTF8String = 'äöüالعَرَبِيَّة'; I assume you mean UTF-16 literal ? 3. PChar on a string literal does not work as expected. You get the bytes of a widestring instead. You should weigh the advantages you outline here against the disadvantages of no longer knowing how string literals will be encoded. It means e.g. the resource string tables will have entries that are UTF16 encoded or entries that are UTF8 encoded, depending on the unit they come from. This is highly undesirable. By forcing everything UTF16 we ensure delphi compatibility (yes it does matter) and we also ensure a uniform set of string tables. Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, 5 May 2017 14:30:32 +0200 (CEST) Michael Van Canneyt wrote: >[...] > > AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings > > instead of UTF8String. Please correct me if I'm wrong. > > > > This has several side effects: > > > > 1. When using a character outside BMP FPC stops with: > > Error: UTF-8 code greater than 65535 found > > For example: > > const Eyes = '👀'; > > > > 2. Assigning a UTF-8 literal to an UTF8String requires a > > widestringmanager. > > For example non ISO-8859-1 chars are mangled: > > var u: UTF8String = 'äöüالعَرَبِيَّة'; > > I assume you mean UTF-16 literal ? Huh? The codepage is utf-8, the string type is utf-8, FPC stores UCS-2, why do you ask about UTF-16? > > 3. PChar on a string literal does not work as expected. You get the > > bytes of a widestring instead. > > You should weigh the advantages you outline here against the disadvantages of > no longer knowing how string literals will be encoded. At the moment string literals are encoded in two different ways depending on codepage, character values, literal format and probably some more attributes I don't know. That often confuses users. IMO it would be less confusing if matching string type and codepage would work without conversion. > It means e.g. the resource string tables will have entries that are UTF16 > encoded > or entries that are UTF8 encoded, depending on the unit they come from. > This is highly undesirable. Ehm, the compiled-in resourcestring tables are AnsiString. AFAIK you need the UTF-8 system codepage to use the full UTF-16 capabilities of the rsj files. > By forcing everything UTF16 we ensure delphi compatibility (yes it does > matter) > and we also ensure a uniform set of string tables. It will be a glory day, when this is accomplished. But some people can't wait that long. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, 5 May 2017, Mattias Gaertner wrote:
On Fri, 5 May 2017 14:30:32 +0200 (CEST)
Michael Van Canneyt wrote:
[...]
> AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings
> instead of UTF8String. Please correct me if I'm wrong.
To make sure I was presenting correct facts, I did some tests.
As a result of the tests, I think the above statement is wrong.
{$codepage utf8}
var
p : pchar;
begin
P:=Pchar('some string literal');
end.
Results in the following assembler:
.globl _$PROGRAM$_Ld1
_$PROGRAM$_Ld1:
.ascii "some string literal\000"
.Le11:
Not widestring as far as I can see ?
To be sure, I added some russian characters:
.Ld1:
.ascii "some string literal \320\272\320\270\321\202\320\260"
.ascii "\320\271\321\201\320\272\320\276\320\263\320\276\000"
Again, not widestring ?
home: >cat u.pp
{$codepage utf8}
var
p : pchar;
begin
P:=Pchar('some string literal китайского');
end.
So, I tried a resourcestring:
.Ld3$strlab:
.short 65001,1
.long 0
.quad -1,30
.Ld3:
.ascii "some more \320\272\320\270\321\202\320\260\320\271\321"
.ascii "\201\320\272\320\276\320\263\320\276\000"
Again, no widestring, as far as I can see.
Michael.___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, May 5, 2017 at 2:53 PM, Mattias Gaertner wrote: > 1. When using a character outside BMP FPC stops with: > Error: UTF-8 code greater than 65535 found > For example: > const Eyes = '👀'; I copy a related post from Lazarus list by myself and Sven Barth. It belongs here: On Fri, May 5, 2017 at 3:56 PM, Sven Barth via Lazarus wrote: > That is mainly due to the compiler not supporting surrogate pairs for the > UTF-8 -> UTF-16 conversion. If it would support them, then there wouldn't be > a problem anymore... That is a serious bug. Getting codepoints right is the absolute minimum requirement for Unicode support. Surrogate pairs are the UTF-16 equivalent of multi-byte codepoints in UTF-8. Now I understand this was not caused by our UTF-8 run-time switch "hack". It is a plain bug in FPC. Is there a plan to fix it? Juha ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, 5 May 2017, Juha Manninen wrote: On Fri, May 5, 2017 at 2:53 PM, Mattias Gaertner wrote: 1. When using a character outside BMP FPC stops with: Error: UTF-8 code greater than 65535 found For example: const Eyes = '👀'; I copy a related post from Lazarus list by myself and Sven Barth. It belongs here: On Fri, May 5, 2017 at 3:56 PM, Sven Barth via Lazarus wrote: That is mainly due to the compiler not supporting surrogate pairs for the UTF-8 -> UTF-16 conversion. If it would support them, then there wouldn't be a problem anymore... That is a serious bug. Getting codepoints right is the absolute minimum requirement for Unicode support. Surrogate pairs are the UTF-16 equivalent of multi-byte codepoints in UTF-8. Now I understand this was not caused by our UTF-8 run-time switch "hack". It is a plain bug in FPC. Is there a plan to fix it? Incomplete UTF-16 support is a bug. Bugs should always be fixed? Michael.___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
Am 05.05.2017 16:03 schrieb "Juha Manninen" : > > On Fri, May 5, 2017 at 2:53 PM, Mattias Gaertner > wrote: > > 1. When using a character outside BMP FPC stops with: > > Error: UTF-8 code greater than 65535 found > > For example: > > const Eyes = '👀'; > > I copy a related post from Lazarus list by myself and Sven Barth. > It belongs here: > > On Fri, May 5, 2017 at 3:56 PM, Sven Barth via Lazarus > wrote: > > That is mainly due to the compiler not supporting surrogate pairs for the > > UTF-8 -> UTF-16 conversion. If it would support them, then there wouldn't be > > a problem anymore... > > That is a serious bug. Getting codepoints right is the absolute > minimum requirement for Unicode support. Surrogate pairs are the > UTF-16 equivalent of multi-byte codepoints in UTF-8. > > Now I understand this was not caused by our UTF-8 run-time switch > "hack". It is a plain bug in FPC. > Is there a plan to fix it? Now it is fixed :D (revision 36116; maybe we should merge that to fixes once I or someone else tested a big endian target) Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, 5 May 2017 15:55:32 +0200 (CEST) Michael Van Canneyt wrote: > On Fri, 5 May 2017, Mattias Gaertner wrote: > > > On Fri, 5 May 2017 14:30:32 +0200 (CEST) > > Michael Van Canneyt wrote: > > > >> [...] > >> > AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings > >> > instead of UTF8String. Please correct me if I'm wrong. > > To make sure I was presenting correct facts, I did some tests. > > As a result of the tests, I think the above statement is wrong. Naah, not wrong, just a non precise term "UTF-8 string literal". ;) ASCII is stored by FPC as 8-bit string. No problem with that. The interesting part are the non ASCII strings. Try your code with the string examples I gave. Mattias ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, 5 May 2017, Mattias Gaertner wrote: On Fri, 5 May 2017 15:55:32 +0200 (CEST) Michael Van Canneyt wrote: On Fri, 5 May 2017, Mattias Gaertner wrote: > On Fri, 5 May 2017 14:30:32 +0200 (CEST) > Michael Van Canneyt wrote: > >> [...] >> > AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings >> > instead of UTF8String. Please correct me if I'm wrong. To make sure I was presenting correct facts, I did some tests. As a result of the tests, I think the above statement is wrong. Naah, not wrong, just a non precise term "UTF-8 string literal". ;) ASCII is stored by FPC as 8-bit string. No problem with that. The interesting part are the non ASCII strings. Try your code with the string examples I gave. I used non-ascii. Did you not see the russian characters ? Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
Am 05.05.2017 15:55 schrieb "Michael Van Canneyt" : > > > > On Fri, 5 May 2017, Mattias Gaertner wrote: > >> On Fri, 5 May 2017 14:30:32 +0200 (CEST) >> Michael Van Canneyt wrote: >> >>> [...] >>> > AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings >>> > instead of UTF8String. Please correct me if I'm wrong. > > > To make sure I was presenting correct facts, I did some tests. > > As a result of the tests, I think the above statement is wrong. In all three cases you are either explicitly or implicitly forcing the compiler to convert it to Ansi/UTF-8 and since it's a constant it takes a compiletime shortcut. If you'd do a Writeln without the typecast then it will be a UTF-16 constant that is stored in the binary *if* the string contains a character > $7F. Regards, Sven ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
On Fri, 5 May 2017, Sven Barth via fpc-devel wrote: Am 05.05.2017 15:55 schrieb "Michael Van Canneyt" : On Fri, 5 May 2017, Mattias Gaertner wrote: On Fri, 5 May 2017 14:30:32 +0200 (CEST) Michael Van Canneyt wrote: [...] AFAIK FPC stores UTF-8 string literals (-Fcutf8) as widestrings instead of UTF8String. Please correct me if I'm wrong. To make sure I was presenting correct facts, I did some tests. As a result of the tests, I think the above statement is wrong. In all three cases you are either explicitly or implicitly forcing the compiler to convert it to Ansi/UTF-8 and since it's a constant it takes a compiletime shortcut. That was on purpose because Mattias' example on the Lazarus list required this. The point was that PChar() is not usable on string literals. See also his initial mail, which contains the statement: "3. PChar on a string literal does not work as expected. You get the bytes of a widestring instead." So, I did a typecast. (even though I think it is horrible code). If you'd do a Writeln without the typecast then it will be a UTF-16 constant that is stored in the binary *if* the string contains a character $7F. Well, at least now I understand very well why people find it confusing :-) I think we'll need a comprehensive table in the documentation. Can this be produced somehow ? Michael. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPC 3.0.2 for SPARC
On 05/05/17 12:00, Pierre Muller wrote: Hi Mark, Le 05/05/2017 à 13:00, Mark Morgan Lloyd a écrit :> This is something that was discussed on the FPC-Pascal ML but it died.> > I am able to build installation bundles for SPARC running Linux (Debian) > and Solaris (OpenSXCE). The fp IDE works but doesn't have libgdb > support, and I've got limited time to struggle up the learning curve. > Would these be of use for the downloads area, which at present only has > 2.6.2 and 2.4.2 respectively? Yes it would be very nice,libgdb missing for IDE is really not a big deal... Please let me know how you can send me the files. For reasons I'd prefer not to discuss, I can't easily get access to e.g. Dropbox so am limited to email, FTP, or something similarly "old school". If you want to continue this off-list I suggest using the email address in my sig below, since it's less likely to have messages deleted as spam. On Linux we're talking about something like: 53780480 2017-04-07 09:20 fpc-3.0.2.sparc-linux.tar I need to revisit the Solaris build. Checking, I see that I can make fpcsrc without too much trouble using something like gmake NOGDB=1 OPT='-V3.0.0 -O- -gl -Xn -vt' all but the full build is giving me a linking error. Following Pierre's instructions I'm using CHECKLIBGDB=no install/makepack which is eventually failing with /usr/local/src/fpc/fpcbuild-3.0.2/fpcsrc/compiler$ /usr/local/bin/ppcsparc -Ur -Xs -O2 -n -Fusparc -Fusystems -Fu/usr/local/src/fpc/fpcbuild-3.0.2/fpcsrc/rtl/units/sparc-solaris -Fisparc -FE. -FUsparc/units/sparc-solaris -dRELEASE-dsparc -dGDB -dBROWSERLOG -Sew pp.pas /usr/bin/gld:built in linker script:21: syntax error pp.pas(238,36) Error: Error while linking pp.pas(238,36) Fatal: There were 1 errors compiling module, stopping Fatal: Compilation aborted I'm pretty sure I've come across this one before and that the cause is something like the wrong linker being invoked by the makefile, leave it with me so I can go through my notes. I'll be back ;-) - I don't know what the future of SPARC is going to be like, and whether the attempts that are being made to get 64-bit SPARC Linux working will make it any rosier. -- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPC 3.0.2 for SPARC
> gmake NOGDB=1 OPT='-V3.0.0 -O- -gl -Xn -vt' all > > but the full build is giving me a linking error. > > Following Pierre's instructions I'm using > > CHECKLIBGDB=no install/makepack > > which is eventually failing with > > /usr/local/src/fpc/fpcbuild-3.0.2/fpcsrc/compiler$ > /usr/local/bin/ppcsparc -Ur -Xs -O2 -n -Fusparc -Fusystems > -Fu/usr/local/src/fpc/fpcbuild-3.0.2/fpcsrc/rtl/units/sparc-solaris > -Fisparc -FE. -FUsparc/units/sparc-solaris -dRELEASE-dsparc -dGDB > -dBROWSERLOG -Sew pp.pas > > /usr/bin/gld:built in linker script:21: syntax error > pp.pas(238,36) Error: Error while linking > pp.pas(238,36) Fatal: There were 1 errors compiling module, stopping > Fatal: Compilation aborted > > I'm pretty sure I've come across this one before and that the cause is > something like the wrong linker being invoked by the makefile, leave it > with me so I can go through my notes. > > I'll be back ;-) You probably need to add the -Xn option into the makepack script! Pierre ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] FPC 3.0.2 for SPARC
(Personally): AWESOME! This is something that was discussed on the FPC-Pascal ML but it died. I am able to build installation bundles for SPARC running Linux (Debian) and Solaris (OpenSXCE). The fp IDE works but doesn't have libgdb support, and I've got limited time to struggle up the learning curve. Would these be of use for the downloads area, which at present only has 2.6.2 and 2.4.2 respectively? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
Re: [fpc-devel] UTF-8 string literals
> You should weigh the advantages you outline here against the disadvantages of
> no longer knowing how string literals will be encoded.
As a programmer, either I don't want to know (declared const without giving
explicit type) or I do, then I did declare it correctly:
{$codepage utf8}
var u: UTF8String = 'äöüالعَرَبِيَّة';
-> UTF8String containing the characters I entered in the source file (in this
case(!!) just 1:1 copy).
{$codepage utf8}
var u: UCS4String= 'äöü';
-> UCS4 encoded Version, either 00e4 00f6 00fc or the equivalent
with combining characters
There should probably be an error if the characters I typed don't actually exist
in the declared type (emoji in an UCS2String), but otherwise, there's no good
reason why that shouldn't "just work".
> It means e.g. the resource string tables will have entries that are UTF16
> encoded
> or entries that are UTF8 encoded, depending on the unit they come from.
> This is highly undesirable.
Always convert from "unit CP" to UTF8 (or UTF16 if some binary compat is
required), done. Aren't they just internal anyway?
> By forcing everything UTF16 we ensure delphi compatibility (yes it does
> matter)
> and we also ensure a uniform set of string tables.
If that was what happened, ok. But from the error message Matthias listed as (1)
I would assume that the actual string type is UCS2String, at least at some point
in the process.
Just my 2 cents...
Martok
PS: adding to the discussion over on the Lazarus ML: I just found a fourth wiki
page describing a slightly different Unicode support. This is getting
ridiculous.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-devel
